Repository navigation
Regenerate the service/characteristic catalog from Apple metadata - #539
Merged
Merged
Conversation
Regenerates the characteristics/services catalog from Apple's plain-metadata.config (ships with macOS, updated with every OS release) instead of the Xcode HomeKit Accessory Simulator plist, which lags far behind (163 chars / 48 services vs 242 / 69). Mirrors the approach of HAP-NodeJS's generate-definitions.ts: - plain-metadata.config is the primary source (no Xcode required) - the simulator plist is used as an optional overlay (richer permission strings, ValidValues constraints) when installed - scripts/hap_manual_overrides.json holds hand-maintained additions and overrides for things not yet in Apple's machine-readable metadata (seeded with the HKSV 17.99 definitions, incl. the PR #531 content) - names and required/optional lists are stable across regenerations: UUID-based name resolution plus inheritance rules prevent Apple's description drift and required->optional demotions from breaking existing accessories; ValidValues suppress numeric min/max so invalid values are rejected instead of silently clamped - dry-run by default with an add/remove/change report; --write to apply The regenerated catalogs add 108 characteristics and 30 services (AccessCode, Thread, Siri, WebRTC/HKSV 17.99, power management, etc.).
Error out when the HomeKit Accessory Simulator plist is missing instead of silently degrading: without the overlay, ValidValues for newly added characteristics would be missing and permissions fall back to the coarser bitmask. --allow-missing-simulator proceeds explicitly with a loud warning; existing characteristics remain covered by field inheritance.
The Additional Tools simulator plist turned out to work differently
from the old Xcode-bundled one: its Permissions list holds auth flags
(securedRead/securedWrite) and the read/write/notify strings live in
Properties - so mapping Permissions to pr/pw/ev emptied every permission
list, and the plist no longer carries numeric constraints either.
Mirror HAP-NodeJS's generator: permissions and min/max/step come
exclusively from plain-metadata.config; the simulator plist is used
only for enumerated ValidValues ({value: state} reversed to
{state: value}), falling back to inheritance from the previous catalog
per characteristic. Also autodetect the simulator at both install
locations (with and without the Xcode nested Applications folder).
Diffing the regenerated catalog against HAP-NodeJS's generated definitions (V=889) surfaced deliberate fixes they carry in their CharacteristicOverriding config, which our catalog needs too: - WRITE_RESPONSE on the five control-point characteristics (AccessCodeControlPoint, NFCAccessControlPoint, TransitionControl, DataStreamHAPTransport, SetupDataStreamTransport) - uint8 format for HomeKitCameraActive/EventSnapshotsActive/ PeriodicSnapshotsActive - the HKSV 17.99 guide and HAP-NodeJS both define them as uint8 while plain-metadata.config says bool - CurrentTemperature minValue -270 (spec minimum; metadata says 0 which clamps negative readings to 0) - LockLastKnownAction values 9/10 from the newer HAP spec Also regenerated with the simulator plist overlay now active.
- the simulator overlay now contributes ValidValues only (permissions always come from the plain-metadata bitmask), so drop the stale 'coarser bitmask fallback' wording - field inheritance applies per characteristic, not only when the simulator is missing entirely - document the three stability rules (UUID-based naming, required/optional service inheritance, ValidValues suppressing min/max) and the metadata corrections carried in characteristicOverrides
Add full type annotations (catalog/entry/Apple-metadata aliases) to the generator and bring it into the lint scope: - tox.ini: ruff check/format now also cover scripts/gen_hap_types.py (scripts/pickle_to_state.py keeps its pre-existing violations and stays out of scope) - pyproject.toml: add "scripts" to [tool.ty.src] Fixes surfaced by the tools: import sorting, long docstring/signature lines, an overwritten loop variable in the ValidValues conversion, an unused loop variable, and the untyped "_slug_to_name" hack - the slug -> name table is now passed as a separate, properly typed parameter. Generator output is unchanged (dry run reports 0/0/0).
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #539 +/- ##
=======================================
Coverage 90.38% 90.38%
=======================================
Files 26 26
Lines 3796 3796
Branches 458 458
=======================================
Hits 3431 3431
Misses 258 258
Partials 107 107 🚀 New features to boost your workflow:
|
Owner
Author
|
@RaHehl the services/chars are auto-generated from the macOS plist files (see the gen script). I updated the script, re-ran it to get the new services and chars and then added another file of "manual" additions which includes your suff from #531 . I would prefer then to take this PR instead of the other as it contains a more complete pull of all the new things and allows for a clean way to layer additional definitions that apple has not yet shipped (like the HKVS). Thoughts? |
This was referenced Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rewrites
scripts/gen_hap_types.pyto generatecharacteristics.json/services.jsonfrom Apple's machine-readable metadata instead of the Xcode HomeKit Accessory Simulator plist, which lags far behind (163 chars / 48 services) and requires a full Xcode install. This mirrors the approach used by HAP-NodeJS.Sources, in order of precedence:
ValidValues. Required by default,--allow-missing-simulatorto opt out.scripts/hap_manual_overrides.json- hand-maintained file for definitions Apple publishes only in spec documents, so they are added once by hand instead of re-parsing a PDF guide. Seeded with the complete HKSV 17.99 set (supersedes Add HKSV 17.99 camera services and characteristics to the catalog #531), including the ContributingSensors/MotionEnabled additions to MotionSensor and the permission fixes, plus corrections of known metadata quirks that HAP-NodeJS also carries (wr on the control-point characteristics, uint8 formats for the camera operating-mode chars, CurrentTemperature min -270, LockLastKnownAction 9/10).The catalog grows from 163 → 271 characteristics and 48 → 78 services (HKSV 17.99, AccessCode, Thread, Siri, power/firmware management, Wi-Fi transport, …).
Notes