Latency & Benchmark

[AdrianOrenstein]

• Investigate how to get a measure of latency with a display to python pipeline.
• Get a std. Gamestate, capture, input are relatively close.

[XDynames]

I am not convinced the way we found to measure frame latency in FFMPEG is accurate.
Using the frame timestamps with use_wallclock_timestamps=1 gives high readings in the ~300ms range but when viewing previews in open cv I can drive the game without it feeling laggy
In my experience 300ms delay should feel noticeably bad but the displayed frames feel like I am playing the game normally
Have to look into this further

[AdrianOrenstein]

Something to fill in this (now) inaccurate table:

[XDynames]

Yeah it’s very lean now and we might need two: one for inference pipe one for recording pipe

I’d also like to report some Hz values for the different pipes