Need help with GStreamer. Player shows black-screen with other pattern (only ball is ok).

[m-garanin]

Hello,

I encountered the following problem when using GStreamer examples (https://docs.dolby.io/streaming-apis/docs/using-whip-with-gstreamer)
I use first example - OK. Next I try replace "ball" pattern to "snow" (or other).

So pipe is running but player shows Live but only black screen. I repeat this with nvidia-encoder - same result. "ball" - is OK, other patterns - black screen or frozen frame.

==
I write software that uses GStreamer and wanted to add WHIP-streaming to your service. And in the process I encountered the fact that the player only shows the “ball” well, and other content either has a black screen or with large jerks.
Perhaps there are some specific requirements for the format (fps, B-frames,cabac etc)....
Any info will be helpful!

[rnkvogel]

Hello,

I tested the following commands using encoder xh264, qsvh264 and nvh264. With nvh264enc I had to multiply the bitrate much higher and was testing with a NVidia GPU 3070. If I did not set the bit rate the ball would be frozen with nvh264enc.
x264enc
gst-launch-1.0 videotestsrc pattern=snow ! videoconvert ! x264enc bitrate=2500 ! rtph264pay ! 'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' ! whip.sink_0 audiotestsrc wave=sine ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2' ! whip.sink_1 whipsink name=whip auth-token="Dolby.io_RTS_Publishing_Streaem_Token" whip-endpoint="https://director.millicast.com/api/whip/streamName"
qsvh264enc
gst-launch-1.0 videotestsrc pattern=snow ! videoconvert ! qsvh264enc bitrate=2500 ! rtph264pay ! 'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' ! whip.sink_0 audiotestsrc wave=sine ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2' ! whip.sink_1 whipsink name=whip auth-token="Dolby.io_RTS_Publishing_Streaem_Token" whip-endpoint="https://director.millicast.com/api/whip/streamName"
nvh264enc ball works
gst-launch-1.0 videotestsrc pattern=ball ! videoconvert ! nvh264enc bitrate=25000 ! rtph264pay ! 'application/x-rtp,media=video,encoding-name=H264,payload=97,clock-rate=90000' ! whip.sink_0 audiotestsrc wave=sine ! audioconvert ! opusenc ! rtpopuspay ! 'application/x-rtp,media=audio,encoding-name=OPUS,payload=96,clock-rate=48000,encoding-params=(string)2' ! whip.sink_1 whipsink name=whip auth-token="Dolby.io_RTS_Publishing_Streaem_Token" whip-endpoint="https://director.millicast.com/api/whip/streamName"

Both commands tested plays the pattern=ball smooth at 30fps. Both encodes show a b-frames defaulted to 0. I would also recommend in the Dolby.io dashboard to view the stream with the live monitoring to review the data. WHIP with Dolby.io also the options to use additional codecs such as VP8,VP9,AV1 and H265.

[rnkvogel]

I also did test with pattern=snow. and wanted verify the encode used.

x264enc,qsvh264 both streamed successfully. nvh264enc would show a codec error when on GStreamer.
Unexpected response: 400 - Error initializing feed: No common codecs found

[rnkvogel]

I wanted to follow up on the issue and as noted only using encode nvh264enc did not work with pattern=snow and looks to be a plugin issue. All additional patterns for GStreamer using nvh264enc tested did work.

[m-garanin]

I can assume that NVENC automatically selects the encoding mode and maybe it needs to explicitly specify e.g. not to use cabac or GOP -size. So I have a question: is there any requirements for encoding parameters for your service (e.g. YouTube has requirements for gop-size etc).

[rnkvogel]

Encoding with B-frames will create stutter. With the option to use multiple codecs (h264,h265,VP8,VP9,AV1) and stream on non ideal networks, encoding requirements may differ for each.(https://docs.dolby.io/streaming-apis/docs/broadcast#encoding)