iOS app: a native SwiftUI client for FrameOS Cloud and self-hosted backends by mariusandra · Pull Request #724 · FrameOS/frameos

mariusandra · 2026-11-11T03:30:13Z

A native iOS app (SwiftUI, iOS 17+, iPhone and iPad) in ios/. It signs in to FrameOS Cloud by default, or below that to a self-hosted backend, and keeps several accounts side by side. It does on a phone what the workspace does, minus the diagram editor.

From the UI tour on an iOS 27.2 simulator, against a throwaway backend with virtual frames (the store from a local cloud). The last screen is the photo the tour uploaded through the system picker, played as an album: the backend's virtual frame renders it.

What it does

Frames: the list, grouped Active, Inactive and Archived like the website, as large cards or a compact list, with each frame's picture, status word, battery and deploy bar. A frame's page has its picture (tap to fetch a fresh one), render, restart and reboot, deploy (backend), Update FrameOS or firmware, screen on/off, rename and delete. Delete asks the cloud's sudo question natively.
Scenes on a frame: the list with the scene showing, updates and missing API keys marked. Each scene opens its public fields as a form, with Activate or Apply. Also: Update to latest, Remove, grant API keys, the preview row (Install / Discard), and the queued-command list with cancel.
Scenes tab: Store and Saved tabs on both planes, the store on its shelves (the cloud's store, a backend's repositories). Install, or Install & activate, on any frame.
Settings for a frame: what the plane lets it take. On the cloud that is gated by docs/cloud-frames-contract.json; on a backend, Save & deploy. Also the schedule, Button actions, voice-chat switches (frames with a microphone), and cloud-only items: confirm a claimed frame, import its own scenes, local network access with the panel's code, and the API-key and telemetry switches.
Assets: browse, view, share, upload (files and photos), new folder, rename, delete.
Logs and metrics: live over the plane's socket; metrics charted.
Photos: albums, uploads (HEIC turns into JPEG with its metadata kept, everything else byte for byte), rename, move, delete, share links, metadata with location removal, and Play on a frame.
AI chat: the same NDJSON agent loop on both planes, from the AI tab or from a frame's page. The cloud's install proposals are approved in place.
Settings tab: accounts (switch, add, sign out), the account's API keys for scenes, plan and quota.

Signing in with Google or a passkey (a small cloud addition)

The app signs in with email and password through /api/auth/login like the web. For everything else the web can do, Continue with Google and Use a passkey or another way open the cloud's own sign-in page in the system's sign-in sheet (ASWebAuthenticationSession). The cloud then hands the app a session of its own. An app cannot read the browser's cookies, so this needed two new cloud routes (cloud/docs/auth.md "Signing in to the app"):

GET /api/auth/app/authorize?code_challenge=<S256>&state=… sends the person through /login, or straight to Google with provider=google, and comes back to itself.
Signed in, it redirects to frameos://sign-in with a two-minute code: a signed token naming the account and identity by id, with no email in the URL.
POST /api/auth/app/token {code, code_verifier} checks PKCE S256, spends the code once (claimSingleUse), reads the profile fresh, and sets a new session cookie for the app.

Security notes:

Only this app can spend a code. Codes go only to frameos://sign-in, and spending one needs the PKCE verifier only the requesting app holds. The app registers no URL scheme: the sign-in sheet catches the callback itself.
No silent hand-on. Another app could open the same URL in its own sheet. So a browser session from before the request is never handed on silently: /login/app asks "Sign in to the app?" (Continue, Use another account, Cancel). Signing in during the request counts as the consent. Continue is a fetch followed by a navigation, because the CSP's form-action would refuse a frameos: redirect after a form post.
No sudo shortcut. The app's session inherits the browser session's authenticated_at (createSession gains an optional authenticatedAt). So a handoff never counts as a fresh sign-in for sudo mode, and an old browser session can't become a recent one by passing through the app.
Confirming it is you in the app works for accounts without a password or code (Google, passkeys). It uses the same flow with reauth=1: a stale browser session goes through /login/reauth first, and the fresh app session replaces the old one, which is signed out.

Tests:

Cloud unit tests: the RFC 7636 vector, token types, expiry, keys.
An integration suite on Postgres (app-sign-in.integration.test.ts): sign-in, Google, ask-first, origin check, sudo, cancel, bad challenge, deleted account, replay, inherited authenticated_at.
Swift tests for the request and the callback.
CloudIntegrationTests.webSignInHandoff against a real cloud.
A UI test that signs in through the real system sheet on a simulator, both signed out and with Safari already signed in. Google's own pages can't be driven in a test; the sheet, the cloud's page, the code and the exchange are the same for it.

App Store readiness (`docs/ios-app-store.md`)

The guidelines a first submission most often fails on are handled in the app:

Account deletion in Settings (5.1.1(v)).
A consent step before the AI chat sends anything to OpenAI (5.1.2(i)).
"Report this scene" on store scenes (1.2).
No plan or billing links, and no "Pricing" link on the cloud's sign-in pages inside the app's sheet (3.1.3(f)).
Privacy policy and terms links.
A privacy manifest.
ios/bin/release, which archives for devices and uploads to App Store Connect for TestFlight. The device build archives with no warnings.

One decision is left, 4.8. With "Continue with Google" in the app, App Review expects Sign in with Apple as well. Either add it to the cloud (web and app; it needs Apple keys), or ship 1.0 without Google in the app. The doc has both.

How it avoids drift

The app calls the routes the SPA calls, through one transport boundary (ios/FrameOSKit/Sources/FrameOSKit/Core/APIClient.swift, the counterpart of apiFetch). It gets no routes of its own. What the SPA decides in shared code, the app reads from the same sources (docs/ios-app.md has the table):

Generated tables: generate_deploy_progress_contract.py and generate_cloud_contract.py gain a Swift output, checked by the backend suite's --check.
New corpora, replayed by both the SPA's cloud/apps/auth-web/src/test/shared-spa/*-fixtures.test.ts and the app's tests. The expected values were produced by the SPA's own functions:
- docs/frame-status-fixtures.json: status word, the line under a frame's name, deploy bar, firmware word.
- docs/scene-form-fixtures.json: the implicit refresh-interval field and showIf.
- docs/button-map-fixtures.json: the Button actions rows and the rules they write.
- docs/api-path-fixtures.json: backend project scoping.
Existing corpora: server-address-fixtures.json gets its fourth runner, and the reader sections of deploy-progress-fixtures.json replay in Swift.
Read from the SPA's source: the pending-command labels, by a Swift test.
Installs run on the server: a backend's scene and album installs call its own MCP tools (frame_scene_install, photo_album_install), so the id remapping is not copied into Swift. The cloud's installs are single routes.
Installing is consent, on both planes: the SPA grants a store scene the API keys it declares when it installs it. The cloud's /scenes/add takes settings_groups for that, and only grants what the scene declares. The backend's MCP install had no such argument, so a store scene installed through it got none of its keys.
- The backend's frame_scene_install now takes settings_groups and answers declared_settings_groups / granted_settings_groups, the cloud tool's names. Without the argument nothing changes.
- The declared-groups walk the deploy uses is now one helper (scene_declared_setting_groups in models/frame.py), shared by the deploy and the tool. A backend test covers it, and the mypy baseline drops two errors.
- The app passes every group on both planes, so each plane grants exactly what the scene declares.

AGENTS.md gets a section for the app, and docs/api-triality.md notes the second client.

Checking it

ios/bin/test: FrameOSKit's 36 tests (the corpora, the contracts, the client and parsing). They need only the Command Line Tools.
ios/bin/check-ui: type-checks every screen for macOS.
ios/bin/ui-test (new): XCUITest on an iPhone simulator, keeping a screenshot of every screen. CI runs the sign-in tests on every change. Locally, TourUITests taps through the whole app on either plane: sign in, frames, a frame, a store scene searched for and installed, its settings, every frame page, every tab, a new album, a photo through the system picker, its info, and the album played on a frame. Both tours pass on Xcode 27.2, on an iPhone 18 Pro and an iPad Pro 11-inch.
ios/bin/test-backend (new): a throwaway backend from the checkout, with its own SQLite and Redis, a user and two virtual frames.
Signed in to a cloud: CloudIntegrationTests (new) passes against a separate local cloud with its own database, its hub and two virtual frames. It covers a store scene installed, activated and removed, settings, schedule, photos, the live socket (through a one-origin proxy, as nginx does in production) and sign-out. The cloud tour passes against the same instance.
Against a real backend: BackendIntegrationTests passes.
Plain http to any backend: NSAllowsLocalNetworking next to NSAllowsArbitraryLoads made iOS ignore the latter, so a backend at an ordinary hostname over http was refused. Fixed; the tour passes over http://127.0.0.1.nip.io:8997.
Every cloud call checked against its route handler: a separate review found nine mismatches, all fixed.

What the simulator runs found, all fixed:

"Add photos" and "Upload photos" opened nothing: a PhotosPicker inside a Menu does not present on iOS 27.
The Origin header kept a typed cloud address's capitals, so every write was refused as invalid_origin. It is now written like a browser's.
Every photo read as having a color profile, because JSONValue takes nil as a literal (JSON null).
After a search, the keyboard covered "Install & activate".
The frame picture was narrower than its row.
Photo tiles read "Map Pin" to VoiceOver.
Wording: store shelves were raw slugs, "Behaviour" used British spelling, "Zero KB".

Not in the app, deliberately

The diagram editor, code and app editing, scene publishing, adding frames (flashing, SD images, SSH, adoption), hardware wiring, per-photo color profiles, a backend's terminal and build tools. Settings links to the web workspace for those.

Found along the way

The SPA shows "upgrade queued" (FrameDashboardSurface.tsx), a word docs/ui-vocabulary.md bans. The app says "update queued". Not changed in the SPA here.
An album on a backend's virtual frame: the frame cannot load the photos when the backend's own address is local (127.0.0.1, 192.168.x.x) unless the frame's Local network access is on. Its scene HTTP is kept off the LAN by design (embedded_wasm_render.mjs). With the switch on, the album plays (last screenshot above). This is the same from the web, so it is not changed here. Whether a backend's own share links should be exempt is a decision for you.

🤖 Generated with Claude Code

…ckends ios/ is a SwiftUI app (iOS 17+) that signs in to FrameOS Cloud by default (email, password, the second factor) or, below it, to a self-hosted backend, and does on a phone what the workspace does without the diagram editor: frames with their pictures, status and deploy bar; scenes on a frame with their public fields (Activate / Apply), install from the store or the backend's catalog, update, remove, the preview row; render, restart, reboot, deploy, firmware updates, the command queue; settings, schedule, button actions, voice permissions; assets; logs and metrics; the photo library with uploads, albums, share links and "play on a frame"; the AI chat; several accounts at once. It calls the routes the shared SPA calls, through one transport boundary (APIClient, the counterpart of apiFetch: project scoping on a backend, the Origin header on the cloud). What the SPA decides in shared code, the app decides from the same sources, so the two cannot drift silently: - generated tables: generate_deploy_progress_contract.py and generate_cloud_contract.py gain a Swift output (the backend suite's --check covers them); - new fixture corpora replayed by both the SPA (cloud shared-spa tests) and the app: frame-status (status word, activity line, deploy bar, firmware word), scene-form (implicit refresh interval, showIf), button-map (Button actions rows), api-path (project scoping); server-address and deploy-progress gain an iOS runner; - the pending-command labels are read from the SPA's source by a test; - a backend's scene and album installs call its own MCP tools, so the id remapping stays in one place. FrameOSKit (no UI) builds and tests without Xcode (ios/bin/test); FrameOSUI type-checks for macOS (ios/bin/check-ui); the new "iOS app" workflow runs the tests and builds the app for the simulator on GitHub's macOS runners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…w SPA runners typed for eslint; workflow lint Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…etwork access, link switches), more tests, a launch check on a simulator Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…s token; an end-to-end test against a real backend and read-only cloud probes; the socket pings at once Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…x tests outside tsc and eslint (bare JSX namespace) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…tings_groups The SPA grants a store scene the service keys it declares when it installs it, on both planes; the cloud's MCP and its /scenes/add route take `settings_groups` for that (granted ∩ declared). The backend's MCP install had no such argument, so a store scene installed through it (the iOS app's backend installs) got none of its keys. It now takes `settings_groups` and answers `declared_settings_groups` / `granted_settings_groups`, the cloud tool's names; without the argument nothing changes. The declared-groups walk the deploy ships keys by is one helper now (scene_declared_setting_groups, scene_is_from_store in models/frame.py), used by shipped_frame_settings and the tool. The app passes every group on both planes, so the plane grants exactly what the scene declares. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-11-11T04:04:34Z

ESP32 firmware size

Built from a543266 by this run; compared with the latest release v2026.11.6.

Image	This PR	v2026.11.6	Δ
esp32-s3 app (OTA image)	3,230,800 _{89.6% of 3520K slot, 365 KB free}	3,230,800	±0
esp32-s3 merged flash image	3,296,336	3,296,336	±0
esp32-s3-32mb app (OTA image)	3,230,800 _{78.3% of 4032K slot, 877 KB free}	3,230,800	±0
esp32-s3-32mb merged flash image	3,361,872	3,361,872	±0

Breakdown by subsystem — `esp32-s3`

Subsystem	This PR	v2026.11.6	Δ
FrameOS core (Nim)	1,117,543	1,117,543	±0
QuickJS	364,558	364,558	±0
Wi-Fi stack	284,079	284,079	±0
fos_* firmware shell (C)	205,527	205,527	±0
Nim stdlib	205,007	205,007	±0
ESP-IDF misc	194,513	194,513	±0
String pool (attributed to efuse)	132,155	132,155	±0
mbedTLS + certificates	124,396	124,396	±0
lwIP / HTTP / WebSocket	123,433	123,433	±0
libc / libm / newlib	93,168	93,168	±0
Storage (SPIFFS / FatFS / SD / NVS)	92,234	92,234	±0
Display drivers (C)	87,429	87,429	±0
FrameOS apps (Nim)	77,014	77,014	±0
Embedded font	57,894	57,894	±0
Nim packages (chrono, zippy, chroma, qrgen, ...)	29,468	29,468	±0
Crypto (monocypher)	19,633	19,633	±0
Total mapped flash	3,208,051	3,208,051	±0

Inside each subsystem

FrameOS core (Nim) — 1,117,543

Part	This PR	v2026.11.6	Δ
`pixie/src`	375,923	375,923	±0
`frameos/js_runtime`	155,580	155,580	±0
`frameos/utils`	148,518	148,518	±0
`frameos/interpreter.nim`	112,337	112,337	±0
`frameos/ui`	73,943	73,943	±0
`system/sceneSettings`	32,725	32,725	±0
`embedded_main.nim`	25,956	25,956	±0
`embedded_runtime.nim`	22,116	22,116	±0
`frameos/event_loop.nim`	19,119	19,119	±0
`frameos/input_state.nim`	14,179	14,179	±0
`frameos_nim_glue.c.obj`	11,911	11,911	±0
`system/sceneSwitcher`	11,717	11,717	±0
`lib/tz.nim`	10,529	10,529	±0
`frameos/single_scene_host.nim`	9,976	9,976	±0
`frameos/planner.nim`	9,529	9,529	±0
`frameos/types.nim`	8,900	8,900	±0
`frameos/values.nim`	7,642	7,642	±0
`frameos/scene_rhythm.nim`	7,229	7,229	±0
`embedded_scene.nim`	6,855	6,855	±0
`frameos/apps.nim`	6,459	6,459	±0
`frameos/spool.nim`	5,786	5,786	±0
`frameos/events_gen.nim`	5,088	5,088	±0
`frameos/app_config.nim`	3,614	3,614	±0
`fos_netguard.c.obj`	3,508	3,508	±0
`frameos/event_map.nim`	3,076	3,076	±0
`patched_chroma_names.nim`	2,909	2,909	±0
`frameos/node_config.nim`	2,566	2,566	±0
`frameos/app_capabilities.nim`	2,473	2,473	±0
`frameos/render_probe.nim`	2,389	2,389	±0
`frameos/events.nim`	2,343	2,343	±0
`frameos/reboot_reason.nim`	2,312	2,312	±0
`frameos/refresh_interval.nim`	2,150	2,150	±0
`frameos/loading_mark.nim`	1,371	1,371	±0
`fos_log_ring.c.obj`	1,269	1,269	±0
`frameos/layout_choice.nim`	1,246	1,246	±0
`frameos/runtime_diagnostics.nim`	767	767	±0
`patched_dragonbox.nim`	743	743	±0
`frameos/render_scale.nim`	678	678	±0
`fos_rows.c.obj`	464	464	±0
`frameos/event_log.nim`	427	427	±0
`fos_version.c.obj`	405	405	±0
`fos_table_probe.nim`	269	269	±0
`frameos/channels.nim`	153	153	±0
`frameos/ids.nim`	120	120	±0
`fos_log_backoff.c.obj`	93	93	±0
`fos_heap_walk.c.obj`	82	82	±0
`frameos/hal`	44	44	±0
`patched_zippy_crc.nim`	28	28	±0
`frameos/retired_apps.nim`	14	14	±0
`frameos/panel_shape.nim`	13	13	±0

QuickJS — 364,558

Part	This PR	v2026.11.6	Δ
`quickjs.c.obj`	291,975	291,975	±0
`libunicode.c.obj`	50,178	50,178	±0
`libregexp.c.obj`	14,007	14,007	±0
`dtoa.c.obj`	5,689	5,689	±0
`cutils.c.obj`	2,141	2,141	±0
`fos_qjs_glue.c.obj`	568	568	±0

Wi-Fi stack — 284,079

Part	This PR	v2026.11.6	Δ
`libnet80211.a`	136,616	136,616	±0
`libpp.a`	61,986	61,986	±0
`libwpa_supplicant.a`	43,697	43,697	±0
`libphy.a`	34,168	34,168	±0
`libesp_wifi.a`	4,616	4,616	±0
`libesp_phy.a`	2,419	2,419	±0
`libesp_coex.a`	291	291	±0
`libcore.a`	283	283	±0
`libespnow.a`	3	3	±0

fos_ firmware shell (C)* — 205,527

Part	This PR	v2026.11.6	Δ
`fos_http.c.obj`	23,385	23,385	±0
`fos_cloud.c.obj`	20,884	20,884	±0
`fos_client.c.obj`	20,687	20,687	±0
`fos_console.c.obj`	17,117	17,117	±0
`fos_scenes.c.obj`	11,866	11,866	±0
`fos_settings.c.obj`	9,366	9,366	±0
`fos_config.c.obj`	6,114	6,114	±0
`fos_ota.c.obj`	6,093	6,093	±0
`fos_cloud_contract.c.obj`	5,578	5,578	±0
`fos_wifi.c.obj`	4,706	4,706	±0
`fos_scene_store.c.obj`	4,674	4,674	±0
`fos_voice.c.obj`	4,076	4,076	±0
`fos_touch.c.obj`	3,655	3,655	±0
`setup_page.html.gz.S.obj`	3,540	3,540	±0
`fos_buttons.c.obj`	3,242	3,242	±0
`fos_assets.c.obj`	3,144	3,144	±0
`fos_panel_draw.c.obj`	2,982	2,982	±0
`fos_config_parse.c.obj`	2,891	2,891	±0
`fos_codec.c.obj`	2,842	2,842	±0
`fos_sd_probe.c.obj`	2,761	2,761	±0
`fos_assets_sd.c.obj`	2,746	2,746	±0
`fos_schedule.c.obj`	2,708	2,708	±0
`fos_events.c.obj`	2,639	2,639	±0
`main.c.obj`	2,615	2,615	±0
`fos_frame_settings.c.obj`	2,613	2,613	±0
`fos_rtc.c.obj`	2,446	2,446	±0
`fos_mic.c.obj`	2,413	2,413	±0
`fos_assets_flash.c.obj`	2,075	2,075	±0
`fos_buzzer.c.obj`	1,959	1,959	±0
`fos_animation.c.obj`	1,929	1,929	±0
`fos_i2c.c.obj`	1,859	1,859	±0
`fos_scene_state.c.obj`	1,817	1,817	±0
`fos_board.c.obj`	1,473	1,473	±0
`fos_url_guard.c.obj`	1,389	1,389	±0
`fos_tz.c.obj`	1,228	1,228	±0
`fos_minisig.c.obj`	1,140	1,140	±0
`fos_battery.c.obj`	1,128	1,128	±0
`fos_led.c.obj`	1,105	1,105	±0
`fos_commands.c.obj`	1,076	1,076	±0
`fos_backend_http.c.obj`	1,064	1,064	±0
`fos_status_screen.c.obj`	873	873	±0
`fos_gauge.c.obj`	841	841	±0
`fos_lan_access.c.obj`	841	841	±0
`fos_wake.c.obj`	805	805	±0
`fos_assets_path.c.obj`	568	568	±0
`fos_framebuffer.c.obj`	513	513	±0
`fos_ota_download.c.obj`	472	472	±0
`fos_voice_binding.c.obj`	441	441	±0
`fos_button_timing.c.obj`	434	434	±0
`fos_slot_cache.c.obj`	369	369	±0
`fos_climate.c.obj`	303	303	±0
`fos_json_guard.c.obj`	267	267	±0
`fos_power.c.obj`	264	264	±0
`fos_printf.c.obj`	261	261	±0
`fos_battery_filter.c.obj`	214	214	±0
`fos_charger.c.obj`	186	186	±0
`fos_wifi_networks.c.obj`	183	183	±0
`fos_spiffs_gc.c.obj`	170	170	±0
`fos_sscanf.c.obj`	154	154	±0
`fos_boot_scene.c.obj`	142	142	±0
`fos_voice_hold.c.obj`	112	112	±0
`fos_spiffs_footprint.c.obj`	89	89	±0

Nim stdlib — 205,007

Part	This PR	v2026.11.6	Δ
`system.nim`	53,916	53,916	±0
`pure/collections/tables.nim`	44,133	44,133	±0
`pure/unicode.nim`	17,431	17,431	±0
`pure/times.nim`	16,413	16,413	±0
`pure/json.nim`	13,678	13,678	±0
`pure/strutils.nim`	7,714	7,714	±0
`pure/parsexml.nim`	5,790	5,790	±0
`pure/base64.nim`	5,103	5,103	±0
`pure/hashes.nim`	4,347	4,347	±0
`pure/xmlparser.nim`	3,407	3,407	±0
`pure/parsejson.nim`	3,392	3,392	±0
`pure/uri.nim`	3,017	3,017	±0
`pure/streams.nim`	2,720	2,720	±0
`std/syncio.nim`	2,334	2,334	±0
`pure/algorithm.nim`	2,307	2,307	±0
`pure/lexbase.nim`	1,775	1,775	±0
`pure/parseutils.nim`	1,744	1,744	±0
`system/exceptions.nim`	1,526	1,526	±0
`pure/strtabs.nim`	1,422	1,422	±0
`pure/pathnorm.nim`	1,204	1,204	±0
`pure/xmltree.nim`	1,202	1,202	±0
`pure/options.nim`	1,080	1,080	±0
`pure/collections/sets.nim`	1,045	1,045	±0
`std/private/ospaths2.nim`	1,024	1,024	±0
`pure/math.nim`	835	835	±0
`pure/collections/deques.nim`	782	782	±0
`pure/strformat.nim`	774	774	±0
`pure/random.nim`	688	688	±0
`std/private/digitsutils.nim`	529	529	±0
`std/private/osdirs.nim`	510	510	±0
`std/monotimes.nim`	497	497	±0
`std/oserrors.nim`	489	489	±0
`system/dollars.nim`	431	431	±0
`std/envvars.nim`	387	387	±0
`std/private/oscommon.nim`	325	325	±0
`std/private/ossymlinks.nim`	187	187	±0
`std/private/decode_helpers.nim`	185	185	±0
`pure/bitops.nim`	162	162	±0
`std/formatfloat.nim`	111	111	±0
`pure/os.nim`	99	99	±0
`std/private/osfiles.nim`	96	96	±0
`core/macros.nim`	89	89	±0
`std/assertions.nim`	87	87	±0
`system/iterators.nim`	20	20	±0

ESP-IDF misc — 194,513

Part	This PR	v2026.11.6	Δ
`libesp_hw_support.a`	36,962	36,962	±0
`libhal.a`	22,252	22,252	±0
`libfreertos.a`	17,447	17,447	±0
`libesp_system.a`	13,503	13,503	±0
`libesp_driver_uart.a`	11,565	11,565	±0
`libheap.a`	8,893	8,893	±0
`libesp_driver_i2c.a`	8,365	8,365	±0
`libesp_lcd.a`	7,737	7,737	±0
`libesp_driver_spi.a`	7,600	7,600	±0
`libconsole.a`	5,745	5,745	±0
`libjson.a`	5,421	5,421	±0
`libesp_driver_gpio.a`	4,760	4,760	±0
`liblog.a`	4,528	4,528	±0
`libxtensa.a`	3,691	3,691	±0
`libbootloader_support.a`	3,498	3,498	±0
`libesp_driver_usb_serial_jtag.a`	3,344	3,344	±0
`libesp_ringbuf.a`	3,322	3,322	±0
`libesp_event.a`	3,095	3,095	±0
`libesp_driver_ledc.a`	2,971	2,971	±0
`libsoc.a`	2,648	2,648	±0
`libesp_mm.a`	2,435	2,435	±0
`libapp_update.a`	2,368	2,368	±0
`libesp_rom.a`	2,364	2,364	±0
`libesp_adc.a`	2,273	2,273	±0
`libesp_timer.a`	2,237	2,237	±0
`libesp_psram.a`	2,056	2,056	±0
`libpthread.a`	1,397	1,397	±0
`libesp_security.a`	1,042	1,042	±0
`libesp_app_format.a`	507	507	±0
`libxt_hal.a`	437	437	±0
`libesp_common.a`	26	26	±0
`libesp_pm.a`	24	24	±0

String pool (attributed to efuse) — 132,155

Part	This PR	v2026.11.6	Δ
`esp_efuse_utility.c.obj`	131,119	131,119	±0
`esp_efuse_rtc_calib.c.obj`	556	556	±0
`esp_efuse_table.c.obj`	268	268	±0
`esp_efuse_api.c.obj`	151	151	±0
`esp_efuse_startup.c.obj`	61	61	±0

mbedTLS + certificates — 124,396

Part	This PR	v2026.11.6	Δ
`libmbedcrypto.a`	67,044	67,044	±0
`libmbedtls.a`	43,452	43,452	±0
`libmbedx509.a`	6,755	6,755	±0
`libesp-tls.a`	5,691	5,691	±0
`libesp_https_server.a`	1,454	1,454	±0

lwIP / HTTP / WebSocket — 123,433

Part	This PR	v2026.11.6	Δ
`liblwip.a`	78,465	78,465	±0
`libhttp_parser.a`	10,347	10,347	±0
`libesp_http_server.a`	9,219	9,219	±0
`libtcp_transport.a`	7,196	7,196	±0
`libesp_http_client.a`	6,166	6,166	±0
`libespressif__esp_websocket_client.a`	6,056	6,056	±0
`libesp_netif.a`	5,984	5,984	±0

libc / libm / newlib — 93,168

Part	This PR	v2026.11.6	Δ
`libc.a`	53,207	53,207	±0
`libm.a`	31,559	31,559	±0
`libnewlib.a`	6,661	6,661	±0
`libstdc++.a`	1,476	1,476	±0
`libgcc.a`	219	219	±0
`libcxx.a`	46	46	±0

Storage (SPIFFS / FatFS / SD / NVS) — 92,234

Part	This PR	v2026.11.6	Δ
`libspiffs.a`	20,348	20,348	±0
`libfatfs.a`	19,924	19,924	±0
`libspi_flash.a`	13,730	13,730	±0
`libnvs_flash.a`	13,665	13,665	±0
`libsdmmc.a`	11,948	11,948	±0
`libvfs.a`	6,224	6,224	±0
`libesp_driver_sdspi.a`	3,822	3,822	±0
`libesp_partition.a`	1,890	1,890	±0
`libesp_vfs_console.a`	678	678	±0
`libnvs_sec_provider.a`	5	5	±0

Display drivers (C) — 87,429

Part	This PR	v2026.11.6	Δ
`(everything else)`	24,491	24,491	±0
`lcd_splash.c.obj`	7,844	7,844	±0
`frameos_panel_table.c.obj`	6,899	6,899	±0
`lcd_st7701s_rgb.c.obj`	5,687	5,687	±0
`it8951.c.obj`	4,628	4,628	±0
`frameos_display.c.obj`	3,213	3,213	±0
`EPD_13in3e.c.obj`	1,792	1,792	±0
`DEV_Config_esp.c.obj`	1,570	1,570	±0
`EPD_3in7.c.obj`	1,564	1,564	±0
`EPD_5in79.c.obj`	1,481	1,481	±0
`DEV_Debug.c.obj`	1,309	1,309	±0
`EPD_2in7.c.obj`	1,292	1,292	±0
29 smaller rows	25,659	25,659	±0

FrameOS apps (Nim) — 77,014

Part	This PR	v2026.11.6	Δ
`apps/data/localImage`	9,875	9,875	±0
`apps/render/split`	9,664	9,664	±0
`apps/render/zoomPan`	7,784	7,784	±0
`apps/apps.nim`	5,975	5,975	±0
`apps/render/text`	4,976	4,976	±0
`apps/data/xmlToJson`	4,465	4,465	±0
`apps/render/svg`	4,381	4,381	±0
`apps/render/image`	3,710	3,710	±0
`apps/render/gradient`	3,129	3,129	±0
`apps/logic/setAsState`	2,460	2,460	±0
`apps/data/downloadImage`	2,402	2,402	±0
`apps/data/rotateImage`	2,206	2,206	±0
`apps/data/downloadUrl`	2,081	2,081	±0
`apps/data/qr`	1,874	1,874	±0
`apps/data/newImage`	1,742	1,742	±0
`apps/data/resizeImage`	1,575	1,575	±0
`apps/render/color`	1,361	1,361	±0
`apps/render/opacity`	1,333	1,333	±0
`apps/data/log`	1,324	1,324	±0
`apps/data/prettyJson`	1,218	1,218	±0
`apps/data/clock`	928	928	±0
`apps/data/parseJson`	852	852	±0
`apps/logic/nextSleepDuration`	661	661	±0
`apps/logic/ifElse`	578	578	±0
`apps/logic/breakIfRendering`	460	460	±0

Embedded font — 57,894

Part	This PR	v2026.11.6	Δ
`assets/fonts.nim`	57,894	57,894	±0

Nim packages (chrono, zippy, chroma, qrgen, ...) — 29,468

Part	This PR	v2026.11.6	Δ
`chrono 0.3.1`	13,414	13,414	±0
`zippy 0.10.19`	6,716	6,716	±0
`chroma 1.0.0`	5,242	5,242	±0
`checksums 0.2.1`	3,290	3,290	±0
`flatty 0.3.4`	530	530	±0
`jsony 1.1.5`	148	148	±0
`bumpy 1.1.3`	128	128	±0

Crypto (monocypher) — 19,633

Part	This PR	v2026.11.6	Δ
`monocypher.c.obj`	16,408	16,408	±0
`monocypher-ed25519.c.obj`	3,225	3,225	±0

Largest 30 objects

Object	Subsystem	This PR	v2026.11.6	Δ
`quickjs.c.obj`	QuickJS	291,975	291,975	±0
`esp_efuse_utility.c.obj`	String pool (attributed to efuse)	131,119	131,119	±0
`frameos/interpreter.nim`	FrameOS core (Nim)	112,337	112,337	±0
`frameos/js_runtime/app_runtime.nim`	FrameOS core (Nim)	107,191	107,191	±0
`assets/fonts.nim`	Embedded font	57,894	57,894	±0
`pixie/src/pixie/fileformats/svg.nim`	FrameOS core (Nim)	57,751	57,751	±0
`pixie/src/pixie/fontformats/opentype.nim`	FrameOS core (Nim)	57,425	57,425	±0
`pixie/src/pixie/paths.nim`	FrameOS core (Nim)	54,167	54,167	±0
`nim/lib/system.nim`	Nim stdlib	53,916	53,916	±0
`libunicode.c.obj`	QuickJS	50,178	50,178	±0
`nim/lib/pure/collections/tables.nim`	Nim stdlib	44,133	44,133	±0
`frameos/ui/layout.nim`	FrameOS core (Nim)	43,573	43,573	±0
`frameos/utils/image.nim`	FrameOS core (Nim)	40,803	40,803	±0
`pixie/src/pixie/fileformats/webp.nim`	FrameOS core (Nim)	37,050	37,050	±0
`pixie/src/pixie/images.nim`	FrameOS core (Nim)	36,338	36,338	±0
`system/sceneSettings/scene.nim`	FrameOS core (Nim)	32,725	32,725	±0
`pixie/src/pixie/fileformats/jpeg.nim`	FrameOS core (Nim)	31,489	31,489	±0
`frameos/js_runtime/runtime.nim`	FrameOS core (Nim)	30,830	30,830	±0
`embedded_main.nim`	FrameOS core (Nim)	25,956	25,956	±0
`fos_http.c.obj`	fos_* firmware shell (C)	23,385	23,385	±0
`embedded_runtime.nim`	FrameOS core (Nim)	22,116	22,116	±0
`pixie/src/pixie/fileformats/png.nim`	FrameOS core (Nim)	21,483	21,483	±0
`fos_cloud.c.obj`	fos_* firmware shell (C)	20,884	20,884	±0
`fos_client.c.obj`	fos_* firmware shell (C)	20,687	20,687	±0
`frameos/event_loop.nim`	FrameOS core (Nim)	19,119	19,119	±0
`frameos/utils/asset_colors.nim`	FrameOS core (Nim)	17,930	17,930	±0
`x509_crt_bundle.S.obj`	mbedTLS + certificates	17,928	17,928	±0
`nim/lib/pure/unicode.nim`	Nim stdlib	17,431	17,431	±0
`fos_console.c.obj`	fos_* firmware shell (C)	17,117	17,117	±0
`frameos/utils/status_screen.nim`	FrameOS core (Nim)	16,590	16,590	±0

_{Flash = .text + .rodata from the linker map via esp_idf_size; "Inside each subsystem" splits every bucket one level down (the Nim core by directory, apps by app, packages by package, ESP-IDF by archive) over all objects, not just the largest. "String pool (attributed to efuse)" is the linker's merged string-literal pool for the whole image, not efuse code — see docs/esp32-image-size.md.}

A review of every cloud call against cloud/apps/auth-web found these, now fixed: - a preview installed on the cloud is saved and pushed the way the SPA does it: without origin, button map and layout in the store copy (they ride the row), with the frame's snapshot as its cover, and granted the keys it declares; - the re-confirm sheet sends the code whenever the account takes one (a recovery code included; a two-factor account refuses any password) and says so for a passkey-only account; - creating a store scene waits 120 s (moderation runs first); - "no time zone" is never sent to the cloud (the contract wants one); - path segments are percent-encoded once, not twice; - an error code without words of its own reads as words, not JSON; - an ESP32's button map stops at the contract's 8 rules (the generator now hands the app each profile's own rule); - a photo upload reports the name it was stored under; - a scene that joins a frame with an active layout wears it, on both planes (the SPA's activeFrameLayout / layoutsWornBy). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- FrameOSUITests: the sign-in screen (every CI build, replacing the bare launch check) and a tour of the whole app against a backend or a cloud, a screenshot of every screen. ios/bin/ui-test runs them and copies the screenshots out. - ios/bin/test-backend: a backend from this checkout with its own SQLite and Redis, a first user and two virtual frames; Ctrl-C removes it. - CloudIntegrationTests: the services signed in to a cloud you may write to: frames, a store scene installed, activated and removed, settings, schedule, photos, sign-out. Passed against a local cloud with a hub and virtual frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The cloud compares Origin with its own origins as text. A cloud address typed with a capital letter (Cloud.FrameOS.net, Frames.local) went out as typed and every write was refused as invalid_origin; a default port or an IPv6 address did the same. Found by the signed-in cloud test, which now also opens the live socket when asked (FRAMEOS_IT_CLOUD_LIVE=1, a cloud with the hub on its own origin). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Xcode is on this Mac now, and the UI tour taps through the whole app on both planes (a throwaway backend with virtual frames; a local cloud with its hub and virtual frames). What it found, fixed: - "Add photos" (Photos) and "Upload photos" (Assets) opened nothing: a PhotosPicker inside a Menu does not present on iOS 27. A menu button now opens .photosPicker. - Every photo read as having a color profile: JSONValue takes `nil` as a literal, so `{ $0.isNull ? nil : $0 }` made JSON null of it. Same trap closed in a schedule event's "disabled"; a test pins both. - The frame picture sized itself to its content, narrower than its row; it is now the panel's shape at full width, and a frame that does not say its size takes the picture's shape instead of 4:3. - After a search the keyboard stayed up over the install sheet, hiding "Install & activate". - Store scenes are grouped by the store's shelves, in shelf order, with the SPA's titles (StoreCategories, pinned to storeCategories.ts by a test). - Photo tiles say their name and "has a location" to VoiceOver, not "Map Pin"; album covers are decorative. - Words: Behavior (US spelling), "Render every … seconds", "0 KB" (not "Zero KB"), a frame's own time zone shown even when the phone does not list it, number fields that no longer cut their labels. The tour itself: sign in, frames, a frame, a store scene searched for and installed, the scene's settings, every frame page, every tab, a new album, a photo through the system picker, its info (location), the album played on a frame. ios/bin/ui-test prefers the newest iPhone Pro and waits for it to boot; CI's sign-in test reads the real error text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…e tour CI's UI test step hung for 15 minutes before any test, with no output to say where. The script now names each stage, adds the tour's photo to the simulator only when a tour will run (with 60 s at most: addmedia can hang on a fresh simulator), and the CI step has its own 20-minute limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

FRAMEOS_UITEST_DEVICE now picks any simulator by name, and the tour finds tabs by their symbol and name: an iPad shows them in a bar at the top, not a tab bar. The backend tour passes on an iPad Pro 11-inch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A native app cannot read the browser's cookies, so the browser hands it a session (cloud/docs/auth.md "Signing in to the app"): - GET /api/auth/app/authorize?code_challenge=<S256>&state=… (the app opens it in the system's sign-in sheet) sends the person through /login, or straight to Google with provider=google, and comes back to itself. Signed in, it redirects to frameos://sign-in with a two-minute code: a signed token naming account and identity by id, no email in the URL. - POST /api/auth/app/token {code, code_verifier}: PKCE S256, spent once (claimSingleUse), the profile read fresh, a new session cookie for the app. Every failure is invalid_grant; audited as account.signed_in (app_handoff). Safety: - A browser session from before the request is never handed on silently: /login/app asks "Sign in to the app?" (Continue / Use another account / Cancel). Another app could open the same URL in its own sheet; signing in during the request is the consent. Continue is a fetch, then a navigation: the CSP's form-action would refuse a frameos: redirect. - The app's session inherits the browser session's authenticated_at (createSession's new authenticatedAt), so a handoff never counts as a fresh sign-in for sudo mode. - reauth=1 sends a stale browser session through /login/reauth first: an account without a password or code (Google, passkeys) confirms it is you in the app this way. Tested: unit tests (RFC 7636 vector, token types, expiry, keys) and an integration suite on Postgres (signed out → sign in → code; Google; ask first; origin check; sudo; cancel; bad challenge; deleted account; replay; inherited authenticated_at). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

- "Continue with Google" and "Use a passkey or another way" on the sign-in screen open the cloud's own sign-in page in the system's sign-in sheet (SwiftUI's webAuthenticationSession, Safari's cookies shared) and trade the code it sends back to frameos://sign-in, with a PKCE verifier only the app holds, for a session of the app's own (CloudWebSignIn, AuthService.cloudFinishWebSignIn). - Confirming it is you: an account without a password or code (Google, passkeys) goes through the same sheet with reauth=1; the fresh session replaces the app's and the old one is signed out. - Tests: the RFC 7636 vector, the URL the sheet opens, every callback case; CloudIntegrationTests runs the handoff against a real cloud (asked first, Continue, exchange, a working session, replay refused); the UI test testCloudWebSignIn signs in through the real sheet on a simulator, both signed out and with Safari already signed in. - NSAllowsLocalNetworking is gone from the ATS settings: beside it iOS ignores NSAllowsArbitraryLoads, so plain http reached only local addresses and a backend at an ordinary hostname was refused. The tour passes over http://127.0.0.1.nip.io:8997. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/frame-status-fixtures.json gains `expect.active` (frameIsActive) on every case and a `groups` section (groupFramesByStatus: Active, Inactive, Archived, each by name with numbers in order and case ignored, ties by id), generated by running the SPA's own functions. The SPA's runner checks both; the app's ports (FrameStatus.isActive, FrameGroups) replay them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

App Store guideline 3.1.3(f) lets a free companion to a paid web service skip in-app purchase only without calls to buy outside the app. The sign-in pages the app's sheet shows (/login and /login/reauth on their way back to /api/auth/app/authorize, and /login/app) leave "Pricing" out of the footer (AuthCard inApp → LegalFooter hidePricing). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…e readiness - Frames: grouped Active, Inactive, Archived like the website (FrameGroups), as large cards or a compact list (the toggle at the top right, kept per device). The frame pickers (install, play an album) use the same order. - Scenes: Store and Saved tabs on both planes. The store sits on its shelves in shelf order; a backend's repositories are shelved the same way and its saved scenes are the Saved tab. - App Store guidelines (docs/ios-app-store.md has the table): - 5.1.1(v): Settings → Delete account (the cloud's own route; the password, or a fresh check of who you are plus the typed email). - 5.1.2(i): the AI chat asks once before its first message, saying the words go to OpenAI. - 1.2: "Report this scene" on store scenes (the cloud's report queue). - 3.1.3(f): no plan or billing link (the plan is shown, not sold). - Privacy policy and terms links; PrivacyInfo.xcprivacy (UserDefaults CA92.1; email, photos, user content for app functionality; no tracking). - ios/bin/release: archive for devices and upload to App Store Connect (TestFlight), signed with your team, build number the UTC time; --unsigned checks the device build (it archives clean, no warnings). - The tour covers the list layout and the Saved tab. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mariusandra and others added 7 commits November 11, 2026 04:29

iOS app: docs/ios-app.md and ios/README.md; AGENTS.md section; the ne…

8369df4

…w SPA runners typed for eslint; workflow lint Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

iOS app: cloud frame extras (confirm, the frame's own scenes, local n…

dc62370

…etwork access, link switches), more tests, a launch check on a simulator Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

iOS app: sign-out ends the backend's login session before revoking it…

12c009b

…s token; an end-to-end test against a real backend and read-only cloud probes; the socket pings at once Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Cloud: the frame-status corpus's runner joins the decorators/frame.ts…

290a6d7

…x tests outside tsc and eslint (bare JSX namespace) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

iOS app: Start on boot for a backend frame's scene

3ecb3d2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mariusandra and others added 11 commits November 11, 2026 05:17

Navigation Menu

Repository navigation

iOS app: a native SwiftUI client for FrameOS Cloud and self-hosted backends - #724

iOS app: a native SwiftUI client for FrameOS Cloud and self-hosted backends#724
mariusandra wants to merge 18 commits into
mainfrom
ios-app

mariusandra commented Nov 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 11, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

mariusandra commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

Signing in with Google or a passkey (a small cloud addition)

App Store readiness (docs/ios-app-store.md)

How it avoids drift

Checking it

Not in the app, deliberately

Found along the way

Uh oh!

github-actions Bot commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ESP32 firmware size

Breakdown by subsystem — esp32-s3

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

mariusandra commented Nov 11, 2026 •

edited

Loading

App Store readiness (`docs/ios-app-store.md`)

github-actions Bot commented Nov 11, 2026 •

edited

Loading

Breakdown by subsystem — `esp32-s3`