Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/awareness-follow-up.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Awareness follow-up

The desktop Awareness page currently groups cursor context and opt-in correction suggestions. It does not yet implement application-awareness controls, expression learning, or the complete correction system requested by the user.

## Reference and evidence

A local Type4Me source checkout at upstream revision `f21339e214d65e9bd4d191ff5136513902e7473b` was reviewed. The installed app identifies itself as 3.2.0; equivalence to that checkout has not been established. The checkout is MIT-licensed: retain its copyright and license notice if source or substantial portions are ported. Relevant upstream files are `Sources/Type4Me/Services/IntelliSenseContext.swift`, `UserEditClassifier.swift`, `ExpressionProfileStore.swift`, `BatchCorrectionInference.swift`, and `Sources/Type4MeIntelliSenseCore/IntelliSensePrompt.swift` in https://github.com/joewongjc/type4me.

The reviewed implementation separates app/control context, cursor context, correction detection, and expression learning. It classifies observed edits and rejects sensitive content and changes to facts or negation before treating edits as style evidence. Style profiles hold abstract features and use sample, day-span, consistency and decay thresholds before emitting limited form-only directives. Its batched correction candidates use independent sessions, multiple days and directional confidence. These are observations about that source revision, not claims about an inaccessible server or later app version.

## Required implementation

- Define independently controlled, opt-in learning and capture boundaries; keep secure fields, sensitive apps, uncertain target changes and blacklist entries excluded. Describe when contextual text may be sent to the configured model.
- Extend bounded, locally observed edits to distinguish content changes, lexical corrections, formatting-only edits and ambiguous cases. Do not learn facts, sensitive values or uncertain target edits as style.
- Persist only bounded abstract expression evidence; define migration, corruption handling, retention, clear/reset semantics and concurrency. Require repeated, consistent evidence across days before applying a profile, with clear app/category/global precedence and decay. Ensure style directives cannot override facts, explicit corrections or user instructions.
- Separate correction suggestion detection from style learning; retain explicit user confirmation for learned vocabulary. Cover conflicting candidates and repeated sessions without treating one edit as a stable habit.
- Integrate the controls into the single Awareness page and exercise opt-out, clearing, app exclusion, restart and prompt behavior across supported hosts. Add deterministic classification, persistence, safety, stale-observer and native integration tests before enabling the feature.
23 changes: 23 additions & 0 deletions openless-all/app/e2e/refinements.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,29 @@ async function openShortcuts(page: Page) {
await settleNavigation(page);
}

test('shortcut search remains visible without focus', async ({ page }) => {
await openShortcuts(page);
const search = page.getByRole('searchbox', { name: '搜索命令或按键' });
const colors = await search.evaluate((element) => ({
search: getComputedStyle(element).backgroundColor,
page: getComputedStyle(element.closest('.ol-settings-content') ?? document.body)
.backgroundColor,
}));
expect(colors.search).not.toBe('rgba(0, 0, 0, 0)');
expect(colors.search).not.toBe(colors.page);
});

test('awareness settings have a dedicated advanced page', async ({ page }) => {
await page.goto('/');
await page.getByRole('button', { name: '设置', exact: true }).click();
const dialog = page.getByRole('dialog', { name: '设置', exact: true });
await dialog.getByRole('button', { name: /^(高级功能|实验与扩展)$/ }).click();
await settleNavigation(dialog);
await dialog.locator('[data-ol-advanced-entry="cursorContext"]').click();
await expect(dialog.locator('[data-ol-advanced-page="cursorContext"]')).toBeVisible();
await expect(dialog.getByText('手改学词', { exact: true })).toBeVisible();
});

test('shortcut search and categories live in the flat settings pane; keyboard close preserves exit', async ({
page,
}) => {
Expand Down
8 changes: 8 additions & 0 deletions openless-all/app/src-tauri/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,14 @@ fn main() {
.flag("-fobjc-arc")
.flag("-mmacosx-version-min=14.0")
.compile("openless_playback_reference");
println!("cargo:rerun-if-changed=src/accessibility_guide_macos.m");
cc::Build::new()
.file("src/accessibility_guide_macos.m")
.flag("-fobjc-arc")
.flag("-mmacosx-version-min=14.0")
.compile("openless_accessibility_guide");
println!("cargo:rustc-link-lib=framework=AppKit");
println!("cargo:rustc-link-lib=framework=ApplicationServices");
println!("cargo:rustc-link-lib=framework=Foundation");
println!("cargo:rustc-link-lib=framework=CoreAudio");
let target_arch = std::env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_default();
Expand Down
96 changes: 96 additions & 0 deletions openless-all/app/src-tauri/src/accessibility_guide_macos.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
#import <AppKit/AppKit.h>
#import <ApplicationServices/ApplicationServices.h>

@interface OLAccessibilityDragView : NSView <NSDraggingSource>
@property(nonatomic, strong) NSURL *applicationURL;
@end

@implementation OLAccessibilityDragView
- (NSView *)hitTest:(NSPoint)point {
return [super hitTest:point] ? self : nil;
}
- (NSDragOperation)draggingSession:(NSDraggingSession *)session sourceOperationMaskForDraggingContext:(NSDraggingContext)context {
return NSDragOperationCopy;
}
- (void)mouseDragged:(NSEvent *)event {
if (!self.applicationURL) return;
NSDraggingItem *item = [[NSDraggingItem alloc] initWithPasteboardWriter:self.applicationURL];
NSImage *icon = [[NSWorkspace sharedWorkspace] iconForFile:self.applicationURL.path];
[item setDraggingFrame:NSMakeRect(0, 0, 48, 48) contents:icon];
[self beginDraggingSessionWithItems:@[item] event:event source:self];
}
@end

static NSPanel *guidePanel;
static NSTimer *guideTimer;

static NSString *OLGuideText(NSString *zh, NSString *en) {
return [[[NSLocale preferredLanguages] firstObject] hasPrefix:@"zh"] ? zh : en;
}

static void OLCloseAccessibilityGuide(void) {
[guideTimer invalidate];
guideTimer = nil;
[guidePanel close];
guidePanel = nil;
}

void openless_show_accessibility_guide(void) {
dispatch_async(dispatch_get_main_queue(), ^{
if (AXIsProcessTrusted()) return;
NSURL *url = [NSBundle mainBundle].bundleURL;
if (![url.pathExtension.lowercaseString isEqualToString:@"app"]) return;
if (guidePanel) {
[guidePanel orderFrontRegardless];
return;
}
NSScreen *screen = NSScreen.mainScreen;
if (!screen) return;
NSRect visible = screen.visibleFrame;
NSRect frame = NSMakeRect(NSMidX(visible) - 210, NSMinY(visible) + 20, 420, 138);
guidePanel = [[NSPanel alloc] initWithContentRect:frame
styleMask:NSWindowStyleMaskBorderless | NSWindowStyleMaskNonactivatingPanel
backing:NSBackingStoreBuffered defer:NO];
guidePanel.level = NSFloatingWindowLevel;
guidePanel.floatingPanel = YES;
guidePanel.hidesOnDeactivate = NO;
guidePanel.releasedWhenClosed = NO;
guidePanel.opaque = NO;
guidePanel.backgroundColor = NSColor.clearColor;
guidePanel.hasShadow = YES;
guidePanel.collectionBehavior = NSWindowCollectionBehaviorCanJoinAllSpaces | NSWindowCollectionBehaviorFullScreenAuxiliary;

NSView *content = guidePanel.contentView;
content.wantsLayer = YES;
content.layer.backgroundColor = NSColor.windowBackgroundColor.CGColor;
content.layer.cornerRadius = 14;
content.layer.masksToBounds = YES;
NSButton *close = [NSButton buttonWithImage:[NSImage imageWithSystemSymbolName:@"xmark" accessibilityDescription:OLGuideText(@"关闭", @"Close")]
target:guidePanel action:@selector(close)];
close.frame = NSMakeRect(382, 104, 24, 24);
close.bordered = NO;
[content addSubview:close];
NSTextField *hint = [NSTextField wrappingLabelWithString:OLGuideText(@"将 OpenLess 拖入辅助功能列表,然后打开开关。", @"Drag OpenLess into the Accessibility list, then turn on its switch.")];
hint.frame = NSMakeRect(18, 88, 352, 36);
hint.font = [NSFont systemFontOfSize:13 weight:NSFontWeightMedium];
[content addSubview:hint];

OLAccessibilityDragView *drag = [[OLAccessibilityDragView alloc] initWithFrame:NSMakeRect(16, 16, 388, 62)];
drag.applicationURL = url;
drag.wantsLayer = YES;
drag.layer.backgroundColor = NSColor.controlBackgroundColor.CGColor;
drag.layer.cornerRadius = 10;
NSImageView *icon = [[NSImageView alloc] initWithFrame:NSMakeRect(12, 9, 44, 44)];
icon.image = [[NSWorkspace sharedWorkspace] iconForFile:url.path];
[drag addSubview:icon];
NSTextField *label = [NSTextField labelWithString:@"OpenLess"];
label.frame = NSMakeRect(68, 21, 280, 20);
label.font = [NSFont systemFontOfSize:14 weight:NSFontWeightSemibold];
[drag addSubview:label];
[content addSubview:drag];
[guidePanel orderFrontRegardless];
guideTimer = [NSTimer scheduledTimerWithTimeInterval:0.5 repeats:YES block:^(NSTimer *timer) {
if (AXIsProcessTrusted() || !guidePanel.visible) OLCloseAccessibilityGuide();
}];
});
}
11 changes: 9 additions & 2 deletions openless-all/app/src-tauri/src/commands/permissions_cmds.rs
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,15 @@ pub fn open_system_settings(pane: String) -> Result<(), String> {
std::process::Command::new("open")
.arg(url)
.spawn()
.map(|_| ())
.map_err(|e| e.to_string())
.map_err(|e| e.to_string())?;
if pane == "accessibility" {
extern "C" {
fn openless_show_accessibility_guide();
}
// The AppKit helper schedules its work on the main thread.
unsafe { openless_show_accessibility_guide() };
}
Ok(())
}
#[cfg(target_os = "windows")]
{
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1312,7 +1312,7 @@ export const de: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: 'Sprachverbesserung und Wiedergabeunterdrückung',
cursorContextTitle: 'Cursorkontext und kurze Eingaben',
cursorContextTitle: 'Wahrnehmung',
startupDiagnosticsTitle: 'Startdiagnose',
cursorContextHint:
'Der Cursorkontext ist standardmäßig aktiv. Hier deaktivieren, wenn naher Text nicht mit Überarbeitungsanfragen gesendet werden soll.',
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1350,7 +1350,7 @@ export const en: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: 'Voice enhancement and playback suppression',
cursorContextTitle: 'Cursor context and short-input behavior',
cursorContextTitle: 'Awareness',
startupDiagnosticsTitle: 'Startup diagnostics',
cursorContextHint:
'Cursor context is enabled by default. Disable it if you do not want nearby text sent with polish requests.',
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/es.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1304,7 +1304,7 @@ export const es: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: 'Mejora de voz y supresión de reproducción',
cursorContextTitle: 'Contexto del cursor y frases cortas',
cursorContextTitle: 'Percepción',
startupDiagnosticsTitle: 'Diagnóstico de inicio',
cursorContextHint:
'El contexto del cursor está activado de forma predeterminada. Desactívalo aquí si no quieres enviar el texto cercano con las solicitudes de mejora.',
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/fr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1324,7 +1324,7 @@ export const fr: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: 'Amélioration vocale et suppression de lecture',
cursorContextTitle: 'Contexte du curseur et phrases courtes',
cursorContextTitle: 'Perception',
startupDiagnosticsTitle: 'Diagnostic de démarrage',
cursorContextHint:
'Le contexte du curseur est activé par défaut. Désactivez-le ici si le texte voisin ne doit pas partir avec les demandes de reformulation.',
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/ja.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1276,7 +1276,7 @@ export const ja: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: '音声強調と再生抑制',
cursorContextTitle: 'カーソル文脈と短文の動作',
cursorContextTitle: '認識',
startupDiagnosticsTitle: '起動診断',
cursorContextHint:
'カーソル文脈は初期状態でオンです。近くの文字を推敲リクエストに含めたくない場合は、ここでオフにします。',
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/ko.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1269,7 +1269,7 @@ export const ko: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: '음성 향상과 재생 억제',
cursorContextTitle: '커서 문맥과 짧은 입력',
cursorContextTitle: '인식',
startupDiagnosticsTitle: '시작 진단',
cursorContextHint:
'커서 문맥은 기본적으로 켜져 있습니다. 주변 텍스트를 다듬기 요청에 포함하지 않으려면 여기서 끄세요.',
Expand Down
4 changes: 2 additions & 2 deletions openless-all/app/src/i18n/zh-CN.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1298,7 +1298,7 @@ export const zhCN = {
},
advancedPages: {
voiceEnhancementTitle: '语音增强与播放抑制',
cursorContextTitle: '光标上下文与短句行为',
cursorContextTitle: '感知',
startupDiagnosticsTitle: '启动诊断',
cursorContextHint: '光标上下文默认开启。如果不希望润色请求携带附近文字,可以在这里关闭。',
},
Expand Down Expand Up @@ -2424,7 +2424,7 @@ export const zhCN = {
multimodal: '管理多模态识别的实验性开关。',
debug: '保留调试录音、探测光标上下文和导出日志。',
voiceEnhancement: '配置高级音频处理、降噪与播放回声抑制。',
cursorContext: '配置润色请求使用的光标附近上下文与短句行为。',
cursorContext: '管理光标上下文、短句行为和手改学词。',
startupDiagnostics: '查看启动耗时、后端握手和延迟探测结果。',
},
descriptions: {
Expand Down
2 changes: 1 addition & 1 deletion openless-all/app/src/i18n/zh-TW.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1236,7 +1236,7 @@ export const zhTW: typeof zhCN = {
},
advancedPages: {
voiceEnhancementTitle: '語音增強與播放抑制',
cursorContextTitle: '游標上下文與短句行為',
cursorContextTitle: '感知',
startupDiagnosticsTitle: '啟動診斷',
cursorContextHint: '游標上下文預設開啟。如果不希望潤色請求帶上附近文字,可以在這裡關閉。',
},
Expand Down
3 changes: 2 additions & 1 deletion openless-all/app/src/pages/Overview.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -770,6 +770,7 @@ function SegmentedToggle<T extends string | number>({
}) {
return (
<div
className="ol-overview-segment"
role="group"
aria-label={ariaLabel}
style={{
Expand All @@ -778,7 +779,7 @@ function SegmentedToggle<T extends string | number>({
padding: 2,
borderRadius: 8,
background: 'var(--ol-surface-2)',
border: '0.5px solid var(--ol-line)',
boxShadow: 'inset 0 0 0 0.5px var(--ol-line)',
}}
>
{options.map((option) => {
Expand Down
3 changes: 1 addition & 2 deletions openless-all/app/src/pages/settings/navigation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ export const ADVANCED_PAGES = [
icon: 'search',
titleKey: 'settings.advancedPages.cursorContextTitle',
group: 'text',
inline: true,
},
{
id: 'startupDiagnostics',
Expand Down Expand Up @@ -70,7 +69,7 @@ export function advancedPageGroup(page: AdvancedPage): AdvancedPageGroup | undef
return 'group' in page ? page.group : undefined;
}

// Cursor context and startup diagnostics stay on the advanced root.
// Startup diagnostics stays on the advanced root.
// Voice enhancement opens its own page, then a third screen for the DSP sliders.
export function isInlineAdvancedPage(page: AdvancedPage): boolean {
return 'inline' in page && page.inline;
Expand Down
12 changes: 10 additions & 2 deletions openless-all/app/src/pages/settings/tabs.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { LocalMigrationSection } from './LocalMigrationSection';
import { LocalModelsSection } from './models/LocalModelsSection';
import { LocalModelsNavContext } from './models/modelsNav';
import { DebugToolsSection } from './DebugToolsSection';
import { VocabularyLearningSection } from './VocabularyLearningSection';
import { AudioDspSettings } from './AudioDspSettings';
import { CodingAgentSection } from './CodingAgentSection';
import { ClaudeConsoleSection } from './ClaudeConsoleSection';
Expand Down Expand Up @@ -664,6 +665,7 @@ export function AdvancedTab({
onOpenAsr: () => Promise<void>;
}) {
const { t } = useTranslation();
const platformCaps = usePlatformCaps();
const [visited, setVisited] = useState<AdvancedPageId[]>([]);
useEffect(() => {
if (page) setVisited((current) => (current.includes(page) ? current : [...current, page]));
Expand Down Expand Up @@ -693,7 +695,6 @@ export function AdvancedTab({
{isInlineAdvancedPage(item) ? (
<div className="ol-advanced-settings-inline">
{item.id === 'voiceEnhancement' && <VoiceEnhancementSection />}
{item.id === 'cursorContext' && <CursorContextSection />}
{item.id === 'startupDiagnostics' && <StartupDiagnostics />}
</div>
) : (
Expand Down Expand Up @@ -782,7 +783,14 @@ export function AdvancedTab({
{testingReady && item.id === 'lessComputer' && <CodingAgentSection />}
{testingReady && item.id === 'claudeConsole' && <ClaudeConsoleSection />}
{item.id === 'voiceEnhancement' && <VoiceEnhancementSection />}
{item.id === 'cursorContext' && <CursorContextSection />}
{item.id === 'cursorContext' && (
<>
<CursorContextSection />
{showsCursorContextControl(platformCaps?.platform, detectOS()) && (
<VocabularyLearningSection />
)}
</>
)}
{item.id === 'startupDiagnostics' && <StartupDiagnostics />}
{item.id === 'customSyncServer' && page === item.id && <CustomSyncServerSection />}
{item.id === 'debug' && <DebugToolsSection />}
Expand Down
3 changes: 3 additions & 0 deletions openless-all/app/src/styles/global.css
Original file line number Diff line number Diff line change
Expand Up @@ -1089,6 +1089,9 @@ html[data-ol-conservative-layout='true'] .ol-conservative-scope .ol-select-lite-
/* Segmented control (components/ui/Segmented), shared by every segmented choice in
Settings: a white track with the controls' hairline, a neutral chip that slides under
the selected option (useSelectionMotion), labels in the selection language. */
.ol-overview-segment > button {
--ol-corner-radius: calc(var(--ol-control-radius, 8px) - 2px);
}
.ol-segmented {
position: relative;
isolation: isolate;
Expand Down
7 changes: 4 additions & 3 deletions openless-all/app/src/styles/refinements.css
Original file line number Diff line number Diff line change
Expand Up @@ -151,17 +151,18 @@ button:disabled {
min-width: 0;
height: 36px;
padding: 0 11px;
border: 1px solid var(--ol-line-strong);
border: 0;
border-radius: 8px;
color: var(--ol-ink);
background: var(--ol-surface);
background: var(--ol-surface-2);
box-shadow: inset 0 0 0 1px var(--ol-line-strong);
font: inherit;
font-size: 13px;
}
.ol-settings-search-field:focus-visible {
outline: 2px solid var(--ol-blue-ring);
outline-offset: 1px;
border-color: var(--ol-blue);
box-shadow: inset 0 0 0 1px var(--ol-blue);
}
.ol-shortcuts-toolbar {
display: flex;
Expand Down
Loading