Skip to content

Repositories page: local, GitHub, GitLab and combined views - #35

Merged
meddlin merged 2 commits into
mainfrom
claude/multi-source-repo-views-b427f7
Nov 2, 2026
Merged

meddlin merged 2 commits into
mainfrom
claude/multi-source-repo-views-b427f7

Conversation

@meddlin

@meddlin meddlin commented Nov 2, 2026

Copy link
Copy Markdown
Owner

What changed

The Repositories page now has Combined, Local, GitHub and GitLab tabs (kept in the URL as ?source=), so the app is no longer limited to local clones when hosted on a server.

  • Combined (default) joins each local clone with the GitHub/GitLab repos its remotes point at. One row shows where a project lives (clone, GitHub, GitLab) and what it is missing; clones of the same remote collapse into one row.
  • Local is the existing full table, unchanged (favorites, descriptions, issue counts, community scores, preview pane).
  • GitHub / GitLab list what each host reports for the configured account, with links out; cloned repos open their detail page.
  • GitHub uses the existing GITLIB_GITHUB_TOKEN. GitLab uses a new optional GITLIB_GITLAB_TOKEN (read_api, member projects), or GITLIB_GITLAB_USERNAME for public projects, plus GITLIB_GITLAB_URL if self-hosted.

Why stored lists, not fetched per view

Host lists are saved in a new remote_repo_listings table in gitlib.db and read from there, so a page load never calls GitHub or GitLab and the lists survive restarts.

  • The server re-checks a host in the background when its list is missing or older than GITLIB_REPO_REFRESH_HOURS (default 6, 0 = only the Refresh button), on page load and via a 15-minute timer.
  • Checks send each page's ETag; a 304 reuses the stored page and costs no GitHub quota. Refresh ignores ETags and fetches complete lists.
  • A failed check (bad token, rate limit) keeps the stored list on screen with the error noted and pauses automatic retries for 30 minutes.
  • A stored list for a different host/account (e.g. changed GITLIB_GITLAB_URL) is ignored.

Notes for reviewers

  • Catalog logic lives in catalog.server.ts; catalog.ts only holds the server functions. Exporting logic from the server-function file leaked server-only code into the client bundle, so keep them split.
  • Merging is a pure function in apps/web/src/lib/catalog.ts (union-find over remote keys). Only a clone's preferred remote per host is followed, so a fork's upstream isn't pulled in.
  • gitlib.db is included in backups, so the stored lists (including private repo names) will be too.
  • Not built: detail page or preview pane for remote-only repos, sorting on the remote tabs (alphabetical only), and linking a GitHub/GitLab mirror pair that has no clone with both remotes.

Testing

  • npm test (220 tests) and the web typecheck pass. New tests cover the merge logic, GitHub/GitLab paging and ETag revalidation (with fetch stubbed), the store table, and the full flow: first load, restart, recheck, failure, interval 0.
  • Checked in the browser with no tokens (fallback from clone remotes, setup notices) and with an invalid GitHub token (clear 401 notice, page still works).
  • Not run against a real GitHub or GitLab account; the live listing and 304 paths are covered only by the stubbed tests.

meddlin and others added 2 commits November 2, 2026 13:30
…bined views

The Repositories page now has Combined, Local, GitHub and GitLab tabs (kept in
the URL as ?source=). Combined joins each local clone with the GitHub/GitLab
repos its remotes point at, collapsing clones of the same remote into one row.
Local keeps the existing full table.

- lib/catalog: pure merge of clones with host listings (union-find over remote
  keys; only a clone's preferred remote per host is followed)
- server: list the token user's GitHub repos and GitLab projects (paginated,
  cached for 5 minutes, failures reported per host without breaking the rest)
- GitLab listing uses new optional GITLIB_GITLAB_TOKEN, or a username for
  public projects; contributions.ts now shares the GitLab env helpers
- docs/env/compose updated; tests for the merge and the listing code

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…k in the background

The Repositories page used to fetch the full GitHub and GitLab inventory into a
5-minute in-memory cache, so every restart or expiry cost a full listing.

- Listings are stored per host in a new remote_repo_listings table and read from
  there on every view; no host call per page load, and they survive restarts
- The server re-checks a host when its list is missing or older than
  GITLIB_REPO_REFRESH_HOURS (default 6, 0 = only the Refresh button), on page
  load and via a 15-minute timer; the UI polls while a first fetch runs
- Checks send each page's ETag; a 304 reuses the stored page and costs no
  GitHub quota. Refresh fetches complete lists
- A failed check keeps the stored list on screen with the error and pauses
  automatic retries for 30 minutes; a list for another account/host is ignored
- Catalog logic moves to catalog.server.ts so it stays out of the client bundle
- Tests for the store, ETag paging, stored-listing parsing and the end-to-end
  flow (first load, restart, recheck, failure, interval 0)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@meddlin
meddlin merged commit b4d2ce1 into main Nov 2, 2026
1 check passed
@meddlin
meddlin deleted the claude/multi-source-repo-views-b427f7 branch November 2, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant