Skip to content

Feat/add litellm provider #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
appleboy merged 7 commits into from
Sep 28, 2026
Merged

Feat/add litellm provider #268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
50c10b0
feat: add LiteLLM as AI gateway provider
RheagalFire Jul 16, 2026
6c10e4d
test: add comprehensive edge case tests for LiteLLM provider
RheagalFire Jul 16, 2026
08fbdff
Merge branch 'main' into feat/add-litellm-provider
appleboy Sep 28, 2026
b7b8b9f
Merge branch 'main' into feat/add-litellm-provider
appleboy Sep 28, 2026
e2b73c8
fix(cmd): honor shared helper for LiteLLM
appleboy Sep 28, 2026
c2fc77e
style(litellm): format imports with golangci-lint
appleboy Sep 28, 2026
9510e84
docs: document LiteLLM setup and credentials
appleboy Sep 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 49 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ A CLI tool written in [Go](https://go.dev) that generates git commit messages or
- [How to Change to Groq API Service](#how-to-change-to-groq-api-service)
- [How to Change to Ollama API Service](#how-to-change-to-ollama-api-service)
- [How to Change to OpenRouter API Service](#how-to-change-to-openrouter-api-service)
- [Support for LiteLLM Proxy](#support-for-litellm-proxy)
- [Usage](#usage)
- [CLI Mode](#cli-mode)
- [Change Commit Message Template](#change-commit-message-template)
Expand All @@ -62,7 +63,7 @@ A CLI tool written in [Go](https://go.dev) that generates git commit messages or

## Features

- Supports [Azure OpenAI Service](https://azure.microsoft.com/en-us/products/cognitive-services/openai-service), [OpenAI API](https://platform.openai.com/docs/api-reference), [Gemini][60], [Anthropic][100], [Ollama][41], [Groq][30], and [OpenRouter][50].
- Supports [Azure OpenAI Service](https://azure.microsoft.com/en-us/products/cognitive-services/openai-service), [OpenAI API](https://platform.openai.com/docs/api-reference), [Gemini][60], [Anthropic][100], [Ollama][41], [Groq][30], [OpenRouter][50], and LiteLLM proxy.
- Adheres to the [conventional commits specification](https://www.conventionalcommits.org/en/v1.0.0/).
- Integrates with Git prepare-commit-msg Hook, see the [Git Hooks documentation](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks).
- Allows customization of generated diffs by specifying the number of context lines (default: 3).
Expand Down Expand Up @@ -179,7 +180,7 @@ Add the [feature](https://github.com/kvokka/features/tree/main/src/codegpt) to y

## Configuration

First, create your OpenAI API Key. The [OpenAI Platform](https://platform.openai.com/account/api-keys) allows you to generate a new API Key.
For the default OpenAI provider, first create your OpenAI API key. The [OpenAI Platform](https://platform.openai.com/account/api-keys) allows you to generate a new API Key.

![register](./images/register.png)

Expand All @@ -189,13 +190,13 @@ Set the environment variable `OPENAI_API_KEY`:
export OPENAI_API_KEY=sk-xxxxxxx
```

Alternatively, store your API key in a custom config file:
Alternatively, save your API key in the credential store:

```sh
codegpt config set openai.api_key sk-xxxxxxx
```

This will create a `.codegpt.yaml` file in your home directory ($HOME/.config/codegpt/.codegpt.yaml). The following options are available:
General settings are stored in `$HOME/.config/codegpt/.codegpt.yaml`. API keys saved with `codegpt config set` use the OS keyring, with a file-based fallback at `$HOME/.config/codegpt/.cache/credentials.json`. The following options are available:

| Option | Description |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand All @@ -204,7 +205,7 @@ This will create a `.codegpt.yaml` file in your home directory ($HOME/.config/co
| **openai.api_key_helper** | Shell command to dynamically generate API key (e.g., from password manager or secret service). |
| **openai.api_key_helper_refresh_interval** | Interval in seconds to refresh credentials from `api_key_helper` (default: `900` seconds / 15 minutes). Set to `0` to disable caching. |
| **openai.org_id** | Identifier for this organization sometimes used in API requests. See [organization settings](https://platform.openai.com/account/org-settings). Only for `openai` service. |
| **openai.model** | Default model is `gpt-4o`, you can change to other custom model (Groq or OpenRouter provider). |
| **openai.model** | Default model is `gpt-4o`, you can change to another model, including a model name configured on your LiteLLM proxy. |
| **openai.proxy** | HTTP/HTTPS client proxy. |
| **openai.socks** | SOCKS client proxy. |
| **openai.timeout** | Default HTTP timeout is `10s` (ten seconds). |
Expand All @@ -213,12 +214,16 @@ This will create a `.codegpt.yaml` file in your home directory ($HOME/.config/co
| **openai.temperature** | Default temperature is `1`. See reference [temperature](https://platform.openai.com/docs/api-reference/completions/create#completions/create-temperature). |
| **git.diff_unified** | Generate diffs with `<n>` lines of context, default is `3`. |
| **git.exclude_list** | Exclude file from `git diff` command. |
| **openai.provider** | Default service provider is `openai`, you can change to `azure`. |
| **openai.provider** | Default provider is `openai`; also accepts `azure`, `gemini`, `anthropic`, and `litellm`. |
| **output.lang** | Default language is `en` and available languages `zh-tw`, `zh-cn`, `ja`. |
| **openai.top_p** | Default top_p is `1.0`. See reference [top_p](https://platform.openai.com/docs/api-reference/completions/create#completions/create-top_p). |
| **openai.frequency_penalty** | Default frequency_penalty is `0.0`. See reference [frequency_penalty](https://platform.openai.com/docs/api-reference/completions/create#completions/create-frequency_penalty). |
| **openai.presence_penalty** | Default presence_penalty is `0.0`. See reference [presence_penalty](https://platform.openai.com/docs/api-reference/completions/create#completions/create-presence_penalty). |
| **openai.stream** | Enable streaming output for real-time token display, default is `false`. |
| **litellm.base_url** | LiteLLM proxy API base URL (default: `http://localhost:4000/v1`). Used instead of `openai.base_url` for LiteLLM. |
| **litellm.api_key** | LiteLLM proxy API key (master key or virtual key); also available through `LITELLM_API_KEY`. |
| **litellm.api_key_helper** | Shell command to retrieve a LiteLLM proxy API key dynamically. |
| **litellm.api_key_helper_refresh_interval** | LiteLLM helper cache interval in seconds (default: `900`). Set to `0` to fetch a key every time. |
| **prompt.folder** | Default prompt folder is `$HOME/.config/codegpt/prompt`. |

### Using API Key Helper for Dynamic Credentials
Expand Down Expand Up @@ -277,18 +282,21 @@ codegpt config set gemini.api_key_helper_refresh_interval 600

#### How It Works

1. **First execution**: CodeGPT runs your helper command and caches the API key in `~/.config/codegpt/.cache/` with restrictive permissions (0600)
1. **First execution**: CodeGPT runs your helper command and caches the API key in the credential store (OS keyring with a file-based fallback)
2. **Subsequent executions**: Within the refresh interval, CodeGPT uses the cached key
3. **After expiration**: CodeGPT automatically re-runs the helper command and updates the cache
4. **Security**: Cache files are stored with owner-only read/write permissions
4. **File fallback**: Credentials and helper cache entries use `~/.config/codegpt/.cache/credentials.json` with owner-only read/write permissions (0600)

#### Priority Order

When multiple API key sources are configured, CodeGPT uses this priority:
For the shared OpenAI credentials, CodeGPT uses this priority:

1. `openai.api_key_helper` (if configured)
2. `openai.api_key` (static config)
2. `openai.api_key` in the credential store
3. `OPENAI_API_KEY` environment variable
4. `openai.api_key` in the configuration file

Gemini and LiteLLM check their provider-specific credentials before falling back to shared credentials. See [LiteLLM proxy configuration](#support-for-litellm-proxy) for its full lookup order.

### How to Customize the Default Prompt Folder

Expand Down Expand Up @@ -499,6 +507,37 @@ codegpt config set openai.headers "HTTP-Referer=https://github.com/appleboy/Code
- **HTTP-Referer**: Optional, for including your app in openrouter.ai rankings.
- **X-Title**: Optional, for showing in rankings on openrouter.ai.

### Support for LiteLLM Proxy

Connect CodeGPT to a running LiteLLM proxy using its OpenAI-compatible API. Set `openai.model` to a model name configured on that proxy.

```sh
codegpt config set openai.provider litellm
codegpt config set litellm.base_url http://localhost:4000/v1
codegpt config set litellm.api_key sk-your-proxy-key
codegpt config set openai.model your-model-name
```

Use `litellm.base_url` for the proxy API endpoint, including the `/v1` path; `openai.base_url` does not configure this provider. The default endpoint is `http://localhost:4000/v1`. CodeGPT requires a non-empty API key. You can use `LITELLM_API_KEY` instead of saving a key with `codegpt config set`.

LiteLLM reuses shared settings such as `openai.model`, `openai.timeout`, `openai.max_tokens`, `openai.temperature`, `openai.stream`, `openai.headers`, `openai.proxy`, and `openai.socks`.

To retrieve the proxy key dynamically:

```sh
codegpt config set litellm.api_key_helper "/path/to/get-litellm-key.sh"
codegpt config set litellm.api_key_helper_refresh_interval 600
```

LiteLLM resolves credentials in this order:

1. `litellm.api_key_helper`
2. `litellm.api_key` (credential store, then `LITELLM_API_KEY`, then configuration file)
3. `openai.api_key_helper`
4. `openai.api_key` (credential store, then `OPENAI_API_KEY`, then configuration file)

Each helper uses its own refresh interval: `litellm.api_key_helper_refresh_interval` for the LiteLLM helper, or `openai.api_key_helper_refresh_interval` for the shared helper. Both default to 900 seconds; set the relevant interval to `0` to fetch a key on every call. If the selected helper fails, CodeGPT returns the error instead of using a static key.

## Usage

There are two methods for generating a commit message using the `codegpt` command: CLI mode and Git Hook.
Expand Down
2 changes: 1 addition & 1 deletion cmd/cmd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ const (

// sensitiveConfigKeys lists the config keys that should be stored in the
// secure credential store rather than in the plaintext YAML config file.
var sensitiveConfigKeys = []string{"openai.api_key", "gemini.api_key"}
var sensitiveConfigKeys = []string{"openai.api_key", "gemini.api_key", "litellm.api_key"}

// migrateCredentialsToStore moves any plaintext API keys found in the YAML
// config into the secure credential store and clears them from the config file.
Expand Down
66 changes: 35 additions & 31 deletions cmd/config_list.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,37 +18,41 @@ func init() {

// availableKeys is a map of configuration keys and their descriptions
var availableKeys = map[string]string{
"git.diff_unified": "Number of context lines in git diff output (default: 3)",
"git.exclude_list": "Files to exclude from git diff command",
"git.template_file": "Path to template file for commit messages",
"git.template_string": "Template string for formatting commit messages",
"openai.socks": "SOCKS proxy URL for API connections",
"openai.api_key": "Authentication key for OpenAI API access",
"openai.api_key_helper": "Shell command to dynamically generate API key",
"openai.api_key_helper_refresh_interval": "Interval in seconds to refresh credentials from apiKeyHelper (default: 900)",
"openai.model": "AI model identifier to use for requests",
"openai.org_id": "Organization ID for multi-org OpenAI accounts",
"openai.proxy": "HTTP proxy URL for API connections",
"output.lang": "Language for summarization output (default: English)",
"openai.base_url": "Custom base URL for API requests",
"openai.timeout": "Maximum duration to wait for API response",
"openai.max_tokens": "Maximum token limit for generated completions",
"openai.temperature": "Randomness control parameter (0-1): lower values for focused results, higher for creative variety",
"openai.provider": "Service provider selection ('openai' or 'azure')",
"openai.skip_verify": "Option to bypass TLS certificate verification",
"openai.headers": "Additional custom HTTP headers for API requests",
"openai.api_version": "Specific API version to target",
"openai.top_p": "Nucleus sampling parameter: controls diversity by limiting to top percentage of probability mass",
"openai.frequency_penalty": "Parameter to reduce repetition by penalizing tokens based on their frequency",
"openai.presence_penalty": "Parameter to encourage topic diversity by penalizing previously used tokens",
"openai.stream": "Enable streaming output for real-time token display",
"prompt.folder": "Directory path for custom prompt templates",
"gemini.project_id": "VertexAI project for Gemini provider",
"gemini.location": "VertexAI location for Gemini provider",
"gemini.backend": "Gemini backend (BackendGeminiAPI or BackendVertexAI)",
"gemini.api_key": "API key for Gemini provider",
"gemini.api_key_helper": "Shell command to dynamically generate Gemini API key",
"gemini.api_key_helper_refresh_interval": "Interval in seconds to refresh Gemini credentials from apiKeyHelper (default: 900)",
"git.diff_unified": "Number of context lines in git diff output (default: 3)",
"git.exclude_list": "Files to exclude from git diff command",
"git.template_file": "Path to template file for commit messages",
"git.template_string": "Template string for formatting commit messages",
"openai.socks": "SOCKS proxy URL for API connections",
"openai.api_key": "Authentication key for OpenAI API access",
"openai.api_key_helper": "Shell command to dynamically generate API key",
"openai.api_key_helper_refresh_interval": "Interval in seconds to refresh credentials from apiKeyHelper (default: 900)",
"openai.model": "AI model identifier to use for requests",
"openai.org_id": "Organization ID for multi-org OpenAI accounts",
"openai.proxy": "HTTP proxy URL for API connections",
"output.lang": "Language for summarization output (default: English)",
"openai.base_url": "Custom base URL for API requests",
"openai.timeout": "Maximum duration to wait for API response",
"openai.max_tokens": "Maximum token limit for generated completions",
"openai.temperature": "Randomness control parameter (0-1): lower values for focused results, higher for creative variety",
"openai.provider": "Service provider selection ('openai' or 'azure')",
"openai.skip_verify": "Option to bypass TLS certificate verification",
"openai.headers": "Additional custom HTTP headers for API requests",
"openai.api_version": "Specific API version to target",
"openai.top_p": "Nucleus sampling parameter: controls diversity by limiting to top percentage of probability mass",
"openai.frequency_penalty": "Parameter to reduce repetition by penalizing tokens based on their frequency",
"openai.presence_penalty": "Parameter to encourage topic diversity by penalizing previously used tokens",
"openai.stream": "Enable streaming output for real-time token display",
"prompt.folder": "Directory path for custom prompt templates",
"gemini.project_id": "VertexAI project for Gemini provider",
"gemini.location": "VertexAI location for Gemini provider",
"gemini.backend": "Gemini backend (BackendGeminiAPI or BackendVertexAI)",
"gemini.api_key": "API key for Gemini provider",
"gemini.api_key_helper": "Shell command to dynamically generate Gemini API key",
"gemini.api_key_helper_refresh_interval": "Interval in seconds to refresh Gemini credentials from apiKeyHelper (default: 900)",
"litellm.api_key": "API key for LiteLLM proxy (master key or virtual key)",
"litellm.api_key_helper": "Shell command to dynamically generate LiteLLM API key",
"litellm.api_key_helper_refresh_interval": "Interval in seconds to refresh LiteLLM credentials from apiKeyHelper (default: 900)",
"litellm.base_url": "LiteLLM proxy base URL (default: http://localhost:4000/v1)",
}

// configListCmd represents the command to list the configuration values.
Expand Down
Loading
Loading