A desktop application for creating, browsing, and managing Terminal-Bench-Science benchmark tasks in the Harbor format. Scientists chat with an AI agent (Claude Code CLI) to iteratively generate and refine task files, then validate and test them locally.
Describe the benchmark task you want to create in natural language. A Claude Code agent generates and iterates on all five Harbor files (task.toml, instruction.md, Dockerfile, solve.sh, test.sh) in real time, with tool-call visibility and streaming responses.
Edit Harbor files with a tabbed Monaco editor. The task.toml file has both a structured form view (shown below) and a raw TOML editor. A live chat sidebar keeps the agent conversation accessible while editing.
Discover and import any of the 241 existing tasks from the laude-institute/terminal-bench repository. Search by name or keyword, filter by difficulty and category, preview full task details, and import with one click. Imported tasks are automatically converted from the repo's task.yaml format to Harbor's task.toml format.
Task preview with metadata, instruction, and file list:
Filter by difficulty:
Every agent turn and manual save auto-commits via isomorphic-git. Browse the full commit history, view file-level diffs, and restore any previous version.
- Workspace Tabs — Work on multiple tasks simultaneously
- Harbor Validation — Real-time validation of task files against the Harbor schema
- Critique & Scoring — AI-powered critique across multiple quality dimensions
- Docker Runner — Verify solutions and run agent trials locally (planned)
┌─────────────────────────────────────────────────────────────┐
│ Renderer (React + Tailwind + Zustand) │
│ ┌──────────┬───────────┬──────────┬─────────┬───────────┐ │
│ │ChatPanel │EditorPanel│RunnerPanel│BrowsePanel│HistoryPanel│
│ └──────────┴───────────┴──────────┴─────────┴───────────┘ │
│ ┌──────────────────────────────────────────────────────────┐│
│ │ Zustand stores: ui, workspace, chat, editor, browse ││
│ └──────────────────────────────────────────────────────────┘│
├──────────────── IPC (contextBridge) ─────────────────────────┤
│ Main Process (Node.js) │
│ ┌───────────┬────────────┬──────────┬────────────────────┐ │
│ │AgentService│WorkspaceService│GitService│BrowseService │
│ │(claude CLI)│(CRUD + files) │(isomorphic│(GitHub API + │
│ │ │ │ -git) │ YAML→TOML) │
│ └───────────┴────────────┴──────────┴────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐│
│ │ DatabaseService (JSON file), chokidar file watcher ││
│ └──────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────┘
Key technologies: Electron + electron-vite, React 19, TypeScript, Tailwind CSS v4, Zustand + Immer, Monaco Editor, isomorphic-git, Claude Code CLI (subprocess)
- Node.js >= 18
- npm (or pnpm)
- Claude Code CLI — installed and authenticated (
claudemust be on PATH)
git clone https://github.com/yipihey/TerminalBenchTaskGenerator.git
cd TerminalBenchTaskGenerator
npm install
npm run devThis launches the Electron app with hot-reload for the renderer.
npm run buildOutput is written to out/. Run the packaged app with:
npx electron out/main/index.jsnpm run typechecknode test-e2e.mjs # 20 UI + API tests
node test-browse.mjs # 35 browse-feature testssrc/
├── main/ # Electron main process
│ ├── index.ts # Window creation, IPC registration
│ ├── ipc/ # IPC handlers (one file per domain)
│ │ ├── workspace.ipc.ts
│ │ ├── agent.ipc.ts
│ │ ├── browse.ipc.ts
│ │ ├── git.ipc.ts
│ │ ├── filesystem.ipc.ts
│ │ ├── validation.ipc.ts
│ │ ├── critique.ipc.ts
│ │ └── ...
│ ├── services/ # Business logic
│ │ ├── WorkspaceService.ts
│ │ ├── AgentService.ts
│ │ ├── BrowseService.ts
│ │ ├── GitService.ts
│ │ ├── DatabaseService.ts
│ │ └── ...
│ └── utils/
│ └── paths.ts
├── preload/ # Electron context bridge
│ ├── api.ts # ElectronAPI type definitions
│ └── index.ts # contextBridge exposure
├── renderer/ # React UI
│ ├── components/
│ │ ├── layout/ # AppShell, Sidebar, WorkspaceTabs
│ │ ├── chat/ # ChatPanel, ChatMessage, AgentStatus
│ │ ├── editor/ # EditorPanel, MonacoEditor, TaskTomlEditor
│ │ ├── browse/ # BrowsePanel, TaskCard, TaskPreview
│ │ ├── history/ # VersionHistory
│ │ ├── critique/ # CritiquePanel
│ │ ├── runner/ # RunnerPanel
│ │ └── shared/ # Button, Panel, Badge, Dialog
│ ├── stores/ # Zustand + Immer state
│ │ ├── uiStore.ts
│ │ ├── workspaceStore.ts
│ │ ├── chatStore.ts
│ │ ├── editorStore.ts
│ │ └── browseStore.ts
│ └── hooks/ # Custom React hooks
│ ├── useWorkspace.ts
│ ├── useAgent.ts
│ └── useBrowse.ts
└── shared/
└── types.ts # All TypeScript interfaces
- Create a workspace — generates boilerplate Harbor files
- Chat with the agent — describe your task; Claude Code edits the files in-place via tool calls
- Review in the editor — inspect and manually refine the generated files
- Validate — check the task against Harbor schema rules
- Critique — get AI-powered quality scores and suggestions
- Test — run the solution and verification scripts in Docker
- Click Browse (magnifying glass icon) in the sidebar
- The app fetches the full task index from GitHub (cached for 1 hour)
- Search by name, instruction text, or tags
- Filter by difficulty (easy/medium/hard) or category
- Click a task to preview its full details in the right panel
- Click "Import as Workspace" — downloads all files and converts:
task.yaml→task.toml(Harbor format)instructionfield →instruction.mdsolution.sh→solve.shrun-tests.sh→test.sh(wrapped with Harbor reward output)- All other files copied as-is
Each workspace produces five standard files:
| File | Purpose |
|---|---|
task.toml |
Task metadata: name, domain, difficulty, timeouts, verifier config |
instruction.md |
Natural-language instructions given to the AI agent |
Dockerfile |
Container environment setup |
solve.sh |
Reference solution script |
test.sh |
Verification script (writes 0.0 or 1.0 to /logs/verifier/reward.txt) |
MIT