Skip to content

Feat/s3 file browser v1 - #208

Open
dklOrdix wants to merge 189 commits into
mainfrom
feat/s3-file-browser-v1
Open

dklOrdix wants to merge 189 commits into
mainfrom
feat/s3-file-browser-v1

Conversation

@dklOrdix

@dklOrdix dklOrdix commented Jun 18, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #129

dklOrdix and others added 30 commits May 4, 2026 09:18
…ebar imports

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
…s/utils

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
feat(storage): initial Filebrowser UI implementation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
…ation

S3 File Browser Page Navigation (Pagination)
* #96 disable the storage browser if feature flag is set

* #96 fix svelte-check

* #96 fix lint

* #96 disable storage browser by default
* #102 refactor pagination controls into a separate component

* #102 simplified pagination props
* #93 add modal preview for file display

* #93 fix broken e2e tests

* #93 add preview for parquet files, fix scrollbar visibility in preview, allow ms excel csv data for preview

* #93 improve parquet row fetch handling

* #93 fix lint

* #93 move preview endpoint logic into separate files

* #93 disable parquet preview
* refactor(ObjectTable): extract File and Folde rows into seperate components

* style: format code

* style: use camelCase

* style: use camelCase
* feat: implement single-file download functionality and error handling for S3 storage

* feat: refactor S3 download endpoint to use new API structure and improve error handling

* chore: remove obsolete note about static download route segment in tech debt documentation

* style: format code

* fix: improve type safety in S3 error handling

* fix: preview action

* feat: add context menu header and improve file selection handling in FileExplorer

* fix: reset selection state on context menu close and refine download action logic

* style: format files

* style: remove pointer events from Actions row on ContextMenu

* #91 fix lint

---------

Co-authored-by: F3l1x1vo <fli@ordix.de>
Comment thread src/routes/(app)/storage/page.server.test.ts
Comment thread src/lib/components/storage/modals/PreviewModal.svelte.spec.ts
Comment thread src/lib/storage/archive.svelte.ts Outdated
Comment thread src/lib/components/storage/sidebar/BucketList.svelte.spec.ts
Comment thread e2e/storage/file-operations.spec.ts Outdated
Comment thread src/routes/(app)/api/storage/rename/+server.ts Outdated
Comment thread src/routes/(app)/api/storage/delete/+server.ts
Comment thread src/routes/(app)/api/storage/buckets/+server.ts Outdated
Comment thread docs/DATABASE.md
Comment thread src/architecture/helpers.ts Outdated
/**
* Shared helpers for architecture fitness tests.
*
* ArchUnitTS only scans TypeScript (.ts) source files. Svelte (.svelte) and

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its not

Comment thread eslint.config.js
Comment thread src/lib/components/storage/modals/upload/types.ts Outdated
Comment thread AGENTS.md Outdated
Comment thread src/lib/components/storage/modals/preview/CsvPreview.svelte Outdated
Comment thread src/lib/components/storage/modals/CreateModal.svelte Outdated
Comment thread src/routes/(app)/api/storage/connections/+server.ts Outdated
Comment thread src/lib/server/storage/connections-db.ts Outdated
Comment thread src/lib/storage/errors.spec.ts
Comment thread src/lib/server/storage/encryption.ts
Comment thread src/lib/storage/errors.ts Outdated
Comment thread src/lib/storage/tabs.svelte.ts Outdated
Comment thread src/lib/storage/state.svelte.ts Outdated

@Bohreromir Bohreromir left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please do not mark my comments as resolved, I loose track of them otherwise

Comment thread src/lib/server/auth.ts Outdated
Comment thread src/lib/components/storage/modals/shared/ConflictResolutionDialog.svelte Outdated
Comment thread src/lib/components/storage/modals/shared/ConflictResolutionDialog.svelte Outdated
Comment thread src/lib/storage/connection-store.svelte.ts Outdated
Comment thread src/lib/server/storage/download-tokens.ts Outdated
Comment thread src/routes/(app)/api/storage/download/+server.ts Outdated
Comment thread src/lib/storage/api.test-utils.ts Outdated
Comment thread dev/modules/env.sh Outdated
Comment thread dev/modules/summary.sh Outdated
Comment thread dev/modules/postgresql.sh Outdated
Comment thread dev/modules/garage.sh Outdated
Comment thread src/lib/server/db.ts
@Bohreromir

Bohreromir commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Some things I noticed while testing the UI

Manage Connections

  • Host field accepts any value "%/()$=("?§/("

Bucketview

  • When trying to move a folder into the location where it already is located: Two contradicting toasts appear
image

The folder stayed where it was located while the folder contents got renamed(?)
Another time the folder was renamed too

  • When clearing history/failed operations box the folder nav changes size
  • When creating a new file paste (ctrl+p) does not work in the filename field
  • Fonts look very faint/smeared/weird
image
  • selecting all items, cutting and pasting them into the same location -> select replace -> all files disappear
  • Create a new file, leave name at default. Edit the file and save. Create a new file, leave default name -> previously edited file is overwritten
  • Operations/History Window is open -> right click into file browser opens regular browser right click menu
  • Having a scrolling list and inside only show 8 items
image
  • tabs seem to share what is selected inside of them. If I select items, move into the same folder inside a new tab the selection in the old tab changes

Preview

  • When editing a file so its too large ~600000 lines errors toast is "Failed to save file" -> why not tell user the reason error console shows? (too large)

@F3l1x1vo

Copy link
Copy Markdown
Collaborator

@Bohreromir I've implemented most of your suggestions in 9e2ec68. I have two points to address:

When creating a new file paste (ctrl+p) does not work in the filename field

Ctrl + P is not a valid shortcut. File paste works on Ctrl + V.


Fonts look very faint/smeared/weird

It's the regular bold disabled font we use everywhere. I don't disagree that it appears slightly blurred if you look closely in darkmode but it's clearly legible. This was accepted from the day we first did a mockup with the Stackable Team of the storage browser.

@Bohreromir

Copy link
Copy Markdown
Contributor

@F3l1x1vo Whoops my bad. I meant using ctrl+v into the field for the new file name

@F3l1x1vo

Copy link
Copy Markdown
Collaborator

I meant using ctrl+v into the field for the new file name

There's a conflict there with the regular pasting of text. If the user wants to copy and paste a filename he copied from outside the browser for example, a file in the clipboard of the storage browser would be pasted into that field instead of his text. This pasting of filenames by the clipboard is a very niche and weird interaction that I would suggest to leave out.

@Bohreromir Bohreromir left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some more

Comment thread src/lib/server/db.ts
Comment thread src/lib/storage/schemas.ts Outdated
Comment thread src/lib/server/storage/operations.ts
Comment thread e2e/support/db-migrations.setup.ts
Comment thread src/lib/server/db.ts Outdated
Comment thread docs/DATABASE.md Outdated
return { entries, hasMore: false };
} catch (err) {
throw new Error(
'RAR support requires the "unrar" command to be installed on the server. ' +

@Bohreromir Bohreromir Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't we the ones controlling the server?

Apart from that the unrar license says:
"UnRAR source code may be used in any software to handle
RAR archives without limitations free of charge, but cannot be
used to develop RAR (WinRAR) compatible archiver
and to
re-create RAR compression algorithm, which is proprietary.
Distribution of modified UnRAR source code in separate form
or as a part of other software is permitted, provided that
full text of this paragraph, starting from "UnRAR source code"
words, is included in license, or in documentation if license
is not available, and in source code comments of resulting package."
https://github.com/pmachapman/unrar/blob/master/license.txt

I dont speak lawyer, but that might be something @lfrancke wants to check.

Honestly I'd drop the whole archive thing.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As is the pod sizing limits raw archive size and extracted size together to 120MiB. This doesn't sound a lot to me, especially when shared between users.

Apart from that rar extraction is broken as of now, even with unrar present.
execFileAsync has no encoding option, so stdout is UTF-8-decoded into a string; Buffer.from(stdout, 'binary') (archive.ts:574) then takes the low byte of each char. Any byte ≥ 0x80 becomes U+FFFD and comes back as 0xFD. Works for Text/ASCII, but Parquet inside RARs failes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we have to remove 7zip and rar extraction completely.

maxBytes?: number
): Promise<ArchiveListing> {
const bin = await find7zBinary();
if (!bin) throw new Error('7z support requires 7-Zip to be installed on the server.');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where is this installed? including it in the build has the same rar problem described above as it includes the rar archiever

Comment thread src/lib/storage/clipboard.svelte.ts
Comment thread src/lib/components/storage/sidebar/BucketList.svelte Outdated
const raw = env.STORAGE_ENCRYPTION_KEY;

if (!raw) {
log.error('STORAGE_ENCRYPTION_KEY environment variable is not set');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this .env is missing in .example.env

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in 1468557.

Comment thread docs/DATABASE.md
return { entries, hasMore: false };
} catch (err) {
throw new Error(
'RAR support requires the "unrar" command to be installed on the server. ' +

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we have to remove 7zip and rar extraction completely.

}
}

// ── 7z ───────────────────────────────────────────────────────────────────────

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove 7zip support as we can not ship the requried binary

});
}

// ── RAR ──────────────────────────────────────────────────────────────────────

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove rar support as we can not ship the required binary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Epic: S3 File Browser v1

5 participants