Skip to content

perf(index): group multi-extension languages in the file-type table - #283

Closed
bis3946 wants to merge 1 commit into
MinishLab:mainfrom
bis3946:perf/group-file-extensions
Closed

bis3946 wants to merge 1 commit into
MinishLab:mainfrom
bis3946:perf/group-file-extensions

Conversation

@bis3946

@bis3946 bis3946 commented Oct 4, 2026

Copy link
Copy Markdown

src/semble/index/files.py maps 347 file extensions to 265 languages. 61 of those
languages cover more than one extension, so their names are repeated 82 times
across the table. Since these are module-level string literals, every duplicate is
interned at import — this module is imported by the indexer, the cache layer and
the CLI, and semble typically runs as a long-lived MCP server.

This splits the table in two:

  • _LANGUAGE_EXTENSIONS — the 61 languages reached by 2+ extensions, each named
    once next to its extensions.
  • _SINGLE_EXTENSION_LANGUAGE — the 204 extensions whose language is unique,
    unchanged and in the original row order.

_EXTENSION_TO_LANGUAGE is then built from the two. It is still a dict[str, str]
with the same 347 entries; only the source of the data changed.

Why group these at all: the grouped map is also the honest answer to "which
languages have several extensions?" — currently that is spread over 143 rows.

Verification

  • _EXTENSION_TO_LANGUAGE compared entry-by-entry against main: 347/347 identical.
  • ALL_LANGUAGES, _CODE_LANGUAGES, _DOC_LANGUAGES, _CONFIG_LANGUAGES,
    _DATA_LANGUAGES, _LANGUAGE_TO_EXTENSIONS, _CONTENT_TYPE_LANGUAGES: identical.
  • detect_language() probed on 16 filenames; get_extensions() probed on 5
    content-type combinations: identical output.
  • No behavioural or API change; tests/test_files.py covers detect_language and
    get_extensions and passes unmodified.

Diff: one file, one module-level table. No tests, no other modules, no new deps.

@stephantul

Copy link
Copy Markdown
Contributor

Doesn't follow the contribution guidelines, closing.

@stephantul stephantul closed this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants