Repository navigation
mediad: date frames on the robot clock - #432
Open
Quchaosheng wants to merge 4 commits into
Open
Quchaosheng wants to merge 4 commits into
Quchaosheng wants to merge 4 commits into
Conversation
§11 named abs-capture-time as the mechanism and set finding a receiver as the first job. Nothing fills it: webrtcsink does not, aiortc does not read one, and no browser API carries a capture time. RFC 6051 ntp-64 replaces it, and rtpjitterbuffer already receives that. Record the two images that both carry the jitterbuffer bug that loses the extension under FEC: Yocto 0.15.1+ on GStreamer 1.28.2, and the Debian path's 0.15.3 on 1.26.2. Also correct two claims. rtcp-sync-send-time=false states the frame's timestamp rather than the report's send time; the default adds the configured latency as a constant, not a measured encoding cost. And the section said "built" when only a loopback pair had run, so it now says that, keeps the loopback numbers in their own table, and leaves the robot run as the open item. The check runs from a laptop over an ssh -L tunnel: the board has no Python and no apt. Assisted-by: Codex
A peer reading sr-ntptime got the robot's wall clock, which is 1970 until NTP sets it on a board with no battery-backed RTC, and got the report's send time rather than the frame's. configure_sender_reports sets ntp-time-source=clock-time and rtcp-sync-send-time=false on each consumer's rtpbin. The pipeline runs on GstSystemClock, which is CLOCK_MONOTONIC, the clock t_ns is read from, so the report already states the axis robot.state is on. Both values are looked up before they are written: set_property_from_str panics on a name this plugin series does not have. Assisted-by: Codex
A sender report arrives about once a second, so a peer cannot date any frame before the first one. wire_payloader_setup adds RFC 6051's ntp-64 extension to each consumer's payloader, which puts the buffer timestamp on every packet. rtpsession fills it from the same timestamp the report carries. The extension is registered before webrtcsink picks an id for TWCC, because GstRTPBasePayload holds one extension per id and replaces rather than refuses. A build without rtphdrextntp64 loses the frame dates, not the session. gstreamer-rtp is a separate crate because it is a separate .pc file, so the CI comment and the cross sysroot's module list gain it. Assisted-by: Codex
Prints each decoded frame's GstReferenceTimestampMeta beside the newest robot.state t_ns, so a robot run answers whether the two share a clock and by how much they disagree. Run from a laptop with the robotd socket forwarded, not on the board: the Yocto image has no Python, and the tunnel exercises the real network path. FEC is turned off for this consumer because rtpjitterbuffer drops its RTP-to-NTP mapping whenever the payload type changes, and ULPFEC alternates payload types with H.264 on one SSRC. The negotiation is per consumer, so a browser keeps FEC. Assisted-by: Codex
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #220.
A decoded frame and a
robot.statesample are now on the robot's ownCLOCK_MONOTONIC, so a receiver can relate the two without agreeing on awall clock.
Four commits:
docs: abs-capture-time is a dead end. §11 named that extension as the
mechanism; nothing fills it (webrtcsink, aiortc, browser APIs), and RFC
6051 ntp-64 replaces it. Also records the two images that both carry the
jitterbuffer bug that loses the extension under FEC, and corrects two
claims:
rtcp-sync-send-time=falsestates the frame's timestamp ratherthan the report's send time (the default adds the configured latency as a
constant, not a measured encoding cost), and the section is "built;
verified on loopback", not "built".
mediad: sender reports on CLOCK_MONOTONIC at capture time.
configure_sender_reportssetsntp-time-source=clock-timeandrtcp-sync-send-time=falseon each consumer'srtpbin. Both values arechecked before they are written, because
set_property_from_strpanicson a name this plugin series does not have.
mediad: ntp-64 header extension on every frame.
wire_payloader_setupadds RFC 6051's extension to each consumer'spayloader, so a frame is dated from its first packet instead of from the
first sender report. The extension is registered before webrtcsink picks
an id for TWCC; a build without
rtphdrextntp64loses the frame dates,not the session. Adds the
gstreamer-rtpdependency, the CI packagecomment and the cross-sysroot module.
scripts: check-frame-dating.py. Prints each decoded frame's
GstReferenceTimestampMetabeside the newestrobot.statet_ns. Runsfrom a laptop over an ssh tunnel, since the board has no Python.
Testing
Host build with GStreamer 1.24.2 headers:
The loopback numbers in §11 (2 of 302 frames with FEC on, 300 of 303 with
it off) are the earlier GStreamer 1.26.2 / gst-plugins-rs 0.15.3 run, not a
fresh one. A run on a real robot is what turns that table into numbers;
§11 has the command and the table.
FEC is opted out in the script only, because
rtpjitterbufferdrops itsRTP-to-NTP mapping on a payload-type change and ULPFEC alternates payload
types with H.264 on one SSRC. Negotiation is per consumer, so a browser
keeps FEC.