Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guidance on RPSI implementation #17

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Guidance on RPSI implementation #17

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
42f19c1
Guidance on RPSI implementation
aboba Sep 28, 2023
4620962
Update hevc-webrtc-04.nr
aboba Oct 2, 2023
f65e05c
Forbid use of RPSI as positive acknowledgement
aboba Oct 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions hevc-webrtc-04.nr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,18 @@ not signaled as part of the SDP, H.265 implementations MAY send and
SHOULD support proper interpretation of Display Orientation SEI
messages.

[RFC7798] Section 8.3 specifies the use of the Reference Picture Selection
Indication (RPSI) in H.265. Implementations MUST use the RPSI feedback
message only as a reference picture selection request, and MUST NOT use it
as positive acknowledgement. Receivers that detect that H.265 encoder-decoder
synchronization has been lost SHOULD generate an RPSI feedback message if
support for RPSI has been negotiated, unless the receiver has knowledge that
the sender does not support RPSI. Such knowledge can be established during
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unless the receiver has knowledge that the sender does not support RPSI

Should this be removed? If it was negotiated then the sender supports it, if it wasn't then the assumption is that the sender does not support it. I feel like the "unless" is implied in the first part of the sentence.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stephan pointed out that endpoints can negotiate support for RPSI, yet respond as if it were a PLI. So that is the "unless" part of the SHOULD.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that is different, the encoder may always choose to send a new keyframe in reaction to a RPSI feedback because it determines that is "better"(the more likely case, it has sent a key frame which is more recent than the picture the RPSI feedback references in which case the "MUST act" at the bottom seems inappropriate)?

I just re-read https://www.rfc-editor.org/rfc/rfc7741#section-5.1 which uses a very different notion of RPSI. There, RPSI is used a a positive acknowledgement (presumably on every frame or periodically) whereas we treat it as a PLI replacement? Probably a good thing to discuss in the working group.

One would still need PLIs when being unable to decode a frame for a too large amount of time but want to use RPSI in cases we know that a frame is missing?

Take this series of frames:

1 - 2 - 3 - <missing or incomplete> - 5 - 6

Upon detecting that frame 4 is missing or incomplete (either by receiving the last packet of the frame with the marker bit set or the first packet of frame 5) the receiver would send a RPSI acknowledging frame 3.
The sender could act on that by sending the next frame (with number 22 because of delays) based on 3 as an IRAP.
If it does not the receiver should eventually send a PLI.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, https://www.rfc-editor.org/rfc/rfc7798#section-8.3 to the rescue:

The use of the RPSI feedback message as positive acknowledgement with
HEVC is deprecated.

Should we add this? Can we strengthen it from deprecated to "must not be used" even?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added clarification, copying text from RFC 7798 Section 8.3 and adding a MUST NOT for positive acknowledgement.

capability exchange or through previously sent RPSI requests that were
not replied to by the sender through the use of a non-IRAP picture.
An RTP packet-stream sender that receives an RPSI message MUST act
on that message, and SHOULD change the reference picture.

Unless otherwise signaled, WebRTC implementations that support H.265 MUST encode
and decode pixels with an implied 1:1 (square) aspect ratio.
.in -0.3i
Expand Down