Skip to content

Configure shared object storage - #1422

Merged
eseliger merged 2 commits into
mainfrom
add-shared-object-storage-config
Sep 7, 2026
Merged

eseliger merged 2 commits into
mainfrom
add-shared-object-storage-config

Conversation

@eseliger

@eseliger eseliger commented Sep 7, 2026 •

Copy link
Copy Markdown
Member

Adds documentation and a config template for the Sourcegraph shared object storage, and sets it on all services that require it.

Configure every SOURCEGRAPH_UPLOAD consumer through one shared environment file, defaulting to the bundled blobstore. Document explicit S3 and GCS configuration and credential requirements while retaining legacy precise Code Intel settings.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a0791d-a227-718d-a619-a830f33e95c5
@eseliger
eseliger force-pushed the add-shared-object-storage-config branch from 763389c to a4d1a3d Compare September 7, 2026 00:23
@eseliger eseliger changed the title feat/docker-compose: configure shared object storage Configure shared object storage Sep 7, 2026
Updated the README to clarify the requirements for object storage and the configuration for external S3 or GCS buckets.
@eseliger
eseliger marked this pull request as ready for review September 7, 2026 00:47
@eseliger
eseliger requested review from a team and bobheadxi September 7, 2026 00:48
@eseliger
eseliger merged commit b073f13 into main Sep 7, 2026
2 checks passed
@eseliger
eseliger deleted the add-shared-object-storage-config branch September 7, 2026 10:46
marcleblanc2 pushed a commit that referenced this pull request Nov 2, 2026
…1461)

Follow-up to #1422. `docker-compose.yaml` still set
`PRECISE_CODE_INTEL_UPLOAD_BACKEND=blobstore` and
`PRECISE_CODE_INTEL_UPLOAD_AWS_ENDPOINT=http://blobstore:9000` on five
services that also load `sourcegraph-uploads.env`. A customer reading
the file could reasonably think both sets of settings need changing to
move to S3 or GCS. This removes the legacy lines so
`sourcegraph-uploads.env` is the only object storage configuration in
the compose file.

The removed values did nothing:

- They are the built-in defaults: `PRECISE_CODE_INTEL_UPLOAD_BACKEND`
defaults to `blobstore`
([lsifuploadstore/config.go](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/codeintel/shared/lsifuploadstore/config.go?L31)),
and the endpoint defaults to `http://blobstore:9000` outside the
single-container deployment
([deploy/endpoints.go](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/conf/deploy/endpoints.go)).
- When the legacy backend is `blobstore`,
[`resolveLegacyCompatStorage`](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/object/sourcegraphobjects/backcompat.go?L20-L44)
uses the shared `SOURCEGRAPH_UPLOAD_*` config and keeps only the legacy
bucket name, so the legacy endpoint was never used. All code-intel
upload store consumers (frontend, precise and syntactic code intel
workers, and worker jobs) go through this shim.

Resulting behavior is unchanged:

- Default: bundled blobstore, `lsif-uploads` bucket, as before.
- `sourcegraph-uploads.env` switched to S3/GCS: code-intel uploads go to
the shared bucket under the `lsif-uploads/` prefix, as before.
- Existing deployments that set `PRECISE_CODE_INTEL_UPLOAD_BACKEND=S3`
or `GCS` in an override keep their dedicated bucket.

Search jobs already relied on these defaults; compose never set
`SEARCH_JOBS_UPLOAD_*`.

## Test plan

- Parsed the compose file before and after: the only difference is these
two variables removed from the five services, each of which still has
`env_file: sourcegraph-uploads.env`.
- CI.

Co-authored-by: Amp <amp@ampcode.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants