Repository navigation
Conversation
|
Lo he probado y he podido exportar mis cámaras en HEVC, pero encuentro ciertos problemas por ejemplo el audio algunos si los da y muy mal con un sonido horrible y hay otras que no las da del todo, debería de tener el audio máximo de 24bits que pide apple con aac, y otra cosita que faltan integraciones como Google Nest, Ezviz, entre otros modelos, muestre la calidad real de la cámara si lo tengo en h264 u h265 antes de exportar y las entidades se puedan también exportar junto con la cámara a HomeKit ya sea que tenga luz, sirena entre otros que admita Apple con HAP. |
|
Thanks for the contribution @seydx. I'm also in the middle of a comprehensive rework for HAP-NodeJS to support the new capabilities. Will be good to compare notes as it evolves. Speaking for the Homebridge team...this is likely going to be a methodical process...we're not going to flip the switch here on the new camera capabilities. Just to set people's expectations. There's some forward-planning that's needed from an API perspective for both HAP-NodeJS and Homebridge and we're taking it a step at a time. I write and maintain homebridge-unifi-protect and several other plugins. I've been working through this for a good portion of the summer and testing the capabilities out. Appreciate of the work you've done and look forward to seeing where we're making different choices. |
|
H264 cameras that are 2k cannot take advantage of level 5 since it does not allow it yet only level 4 to 1080p, I have several cameras in H264 camera.ui and the audio does not work it makes a buzzing noise, there is no configuration to change the audio to AAC-ELD or Opus since not even the cameras in H264 and H265 with HKSV3, I get real audio and it is a problem some mute and others with buzz, I have the transmission in HomeKit without audio because they do not work in camera.ui |
|
Just for your information, I'll be pushing my work on CMAF in the next few days. Apple’s current compatibility guide is very incomplete, and I’m in touch with them about it Uploading via CMAF is currently working, and the clips are now visible in Apple Home via this path. However, they can’t be played yet. As soon as that works, I’ll push my work, and then the spec will be complete so you can compare for your rework |
|
@seydx Appreciate it...looking forward to seeing it and look forward to seeing where things can potentially come together. Across our two plugins, we've got a pretty good audience for this capability. I'm expecting to bring the pieces together over the next few weeks. I also expect that some of the bugs/quirks are going to get worked out for iOS 27.2 or thereabouts and will help amp this up a bit. You're a braver soul than I to show your WIP as we move along. 😄 |
|
@seydx @hjdhjd Hey all! I wanted to let you know that I will be uploading to my personal forks for HAP-NodeJS, node-arlo, and homebridge-arlo hopefully later this afternoon/evening. I have taken seydx's work which has been fantastic along with my own work and I have finally gotten a fully functioning HKSV3 model from end to end, Local MultiTieredRTP Streaming, Remote WebRTC Streaming, and CMAF Recording. Two-Way Audio works on both streaming types, although there is an Apple Bug that I have reported that if you use the All Cameras Switcher in the Home App on a Remote WebRTC Streaming and switch between two cameras, you lose two-way audio capabilities until you close the stream and re-open it fresh. The talk button just isn't there until you re-open it. Recordings work for uploading and playback, I had some work that I did for handling the playback in getting the keys and encryption handling correct but I got it and everything looks like it works for me. I'll edit this post and like the repositories and their respective branches as well when I get everything up and confirmed on my end. |
|
I'm glad it helped you and that you solved the CMAF issue! I'll take a closer look at it once you're done :) I'm still waiting here for the official response from Apple, since I haven't looked into that part about the keys and playback any further. As soon as the response comes in, I'll update this PR too, and you can compare your work to it :) |
|
Looking forward to reviewing it as well @ZeliardM - my own work should be pulling most of this all together into something cohesive. I'm expecting a busy November and December to land it all. |
# Conflicts: # src/lib/Accessory.spec.ts # src/lib/Accessory.ts # src/lib/Characteristic.ts # src/lib/Service.ts # src/lib/camera/RecordingManagement.ts # src/lib/camera/index.ts # src/lib/controller/index.ts # src/lib/definitions/CharacteristicDefinitions.spec.ts # src/lib/definitions/ServiceDefinitions.spec.ts # src/lib/definitions/ServiceDefinitions.ts # src/lib/definitions/generator-configuration.ts
Coverage Report for CI Build 41714830934Coverage increased (+0.6%) to 78.896%Details
Uncovered Changes
Coverage Regressions1 previously-covered line in 1 file lost coverage.
Coverage Stats
💛 - Coveralls |
iOS 27 comes with a new set of camera services. Apple calls them "secure video", the hub logs call it HKSV3. This PR adds the definitions for them plus a
SecureVideoControllerthat bundles them behind typed delegates, in the same shape asCameraController.The implementation follows Apple's HomeKit Secure Video Open Source Compatibility Guide for iOS 27, plus a fair amount of iOS and tvOS sysdiagnoses where the guide stops. I used Claude a lot for that part, digging through homed and avconferenced logs, comparing sessions, and turning what the hub actually does into code. The findings below all come from real traffic against a real hub, not from guessing.
What the new path looks like
A secure video accessory has one sensor description (
CameraCapabilities), a global operating mode, motion zones, a multi-tier RTP stream management for local viewing, a WebRTC stream management for remote viewing, and the same HDS recording management the classic HKSV uses. Local viewing is plain SRTP like before, just negotiated through tiers instead of the old start/reconfigure dance. Remote viewing is WebRTC through Apple's relay, with SFrame end-to-end encryption on top of DTLS-SRTP. Recording still goes over HDS bulk send, the hub just analyses it differently.Optional on top of that: CMAF accessory direct upload (buffer, key and client certificate management) where the camera uploads the clip itself instead of streaming it to the hub (see below).
With this path an HEVC camera runs natively end to end. The multi-tier RTP stream for local viewing is the camera's own HEVC, the WebRTC stream for remote viewing is the same HEVC packetized for the relay, and recording is the HEVC stream remuxed to fMP4.
Tested
Note
VCVideoFeatureListStringHelper defaultPayload: featureListStrings is emptyandNo matched feature list string for payload: Apple's H.264 receive path expects a negotiated feature list string (FaceTime heritage), HEVC doesn't need one. Nothing in the SDP or fmtp on the accessory side changes that. So I route H.264 cameras to the classicCameraController, that path is H.264 only in the other direction.CameraRTPStreamManagementon the same accessory don't mix, Home shows "No Response". It's one path or the other per accessory.hvc1sample entry, withhev1the hub aborts after the third fragment. The hub also expects aprftbox in front of eachmoof, otherwise homed crashes while reading the fragment date.WebRTCUpdateSessionare per stream (HKDF with the SSRC as salt before the RFC 9605 derivation, same as the camera's own sender key).Not working yet: CMAF direct upload
The provisioning half is verified against the hub: camera key, signed client certificate, publishing point token, buffer activity and upload commands all arrive as described. The upload itself doesn't. Every PUT to the publishing point (
cloudkit-video-service) returns 400 with an empty body, before authentication kicks in (no client certificate, a self-signed one, other session ids, other bodies, other headers, all 400; other paths 404; GET 401). The hub never talks to that endpoint itself, it only creates the CloudKit clip record, so there's nothing to compare against without a trace of a real HKSV3 camera. The code stays in because the delegate is optional and off unless a consumer wires it, but I wanted to be upfront that this part is unverified. If maintainers would rather see it split out, I can do that.How I test this
I develop and run this in camera.ui, which uses hap-nodejs directly. The HomeKit plugin there carries the WebRTC side (werift), the SFrame crypto, the RTP framing and the media pipeline, and picks the controller by codec: HEVC cameras get the secure video controller, H.264 cameras stay on the classic one.
How you can test
Install camera.ui (https://docs.cameraui.com/install/), add a camera with an HEVC main stream, install the HomeKit plugin from the in-app plugin store and enable HomeKit for that camera. The plugin shows a setup code, add it in Home, then it's the same flow as in the video: live view locally, live view remotely, talk. For recording the hub needs motion events, so the camera needs a motion source in camera.ui, either the built-in detection or a plugin that delivers the camera's own motion events (ONVIF for example). H.264 cameras stay on the classic HKSV path, so to see the new services you need HEVC, or the "Force legacy path" toggle in the camera settings to compare both on the same camera.
hksv3new_1.mp4