Repository navigation
Cast: watch a medialink camera on a TV, with its sound - #4
Merged
Merged
Conversation
cast/medialink_tv.py finds a medialink camera by name and casts its pictures and sound over Miracast to a Roku TV or Windows Wireless Display. CastDisplay.pcm_out() now works on the ESP32-P4 too: it is the board's speaker through the audio pump (castfast.pump_out), which the cast hears. boarddev.pcm_out() there is plain I2S, so its sound never reached the cast. CastDisplay.blit_rect copies whole rows as one run instead of framebuf's pixel-by-pixel blit: 11 ms for 640x480 on the P4 instead of 60.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
cast/medialink_tv.py, which finds a medialink camera on the network by name and casts its pictures and sound to a Roku TV or to Windows Wireless Display over Miracast. The board running it needs no screen. It uses medialink's Video, Screen and Intercom, which reached pydevices main in PyDevices/pydevices#326.Two changes in the cast example make this work on an ESP32-P4:
CastDisplay.pcm_out()now works on the P4. Before, it existed only on a desktop. On the board it returns a PCMOutput over the audio pump (castfast.pump_out), because the cast's tap reads the pump andboarddev.pcm_out()is plain I2S, which the cast never hears. The rate has to matchaudio_format.CastDisplay.blit_rectcopies whole rows as a single slice. framebuf's blit goes pixel by pixel and took 60 ms for a 640x480 picture on the P4. The copy takes 11 ms, which brings a decoded picture'sshow()from about 135 ms down to about 81 ms.What was measured
The camera side was pydevices' surveillance camera example running on a computer, with a folder of numbered JPEG cards and a file of known speech standing in for the microphone. Sound was scored by recording it and transcribing it.
On the Roku, the media player reported
state=play, pcm sound and H.264 video. Wireless Display window shots showed the numbered cards.The rates, and why
Clean sound comes first. With the camera at 10 pictures a second, the P4 showed about 6.5 a second, dropped packets, and sound ran 250–340 ms late. At 5 a second it keeps up with almost no drops. The cast cap (FPS) made no difference to the sound at 30, 15 or 10, since castif only sends when the picture changes, so it is 10. Both rates and the reason are in the docstring.