All geek questions in one place

How to use FFmpeg H264 encoder in WebRTC?

For encoding H264, WebRTC uses OpenH264, which does not support hardware acceleration. There are many third-party codecs included with WebRTC, including WebRTC. How can FFmpeg be used instead? "is_component_ffmpeg = true" seems to do nothing.

The goal is to encode hardware acceleration to reduce latency and CPU utilization. We have a hardware encoder, but I don’t know how to connect it to webrtc. Using hardware acceleration is the closest option.

Where do we need to use ffmpeg? or use h264 data stream with external encoding?

+5
source share
2 answers

We completed the modification of h264_encoder_impl , replacing all OpenH264 API calls with our own encoder calls.

WebRTC constantly continues to ask the encoder implementation to update the bitrate and frame rate, as it considers the current available bandwidth necessary. The HW encoder we used only supported bit rate updates on the fly, and this worked great with WebRTC. Framerate has been set to a fixed value.

Since we did not change the frame rate in accordance with the wishes of WebRTC and still worked fine, I think that the encoded stream can also be sent as soon as RTPFragmentation is correct for this encoded buffer.

+3
source

In the past, we tried to bypass the encoding part of the WebRTC project with little luck (we wanted to transfer data that had already been encoded for several WebRTC clients). I got the impression that it is very closely integrated with the quality of service. WebRTC wants to configure encoder settings based on current network traffic.

The best solution we found was to actually run our own WebRTC using the dtlssrtpenc , nicesink and nicesrc in the OpenWebRTC project:

https://github.com/EricssonResearch/openwebrtc-gst-plugins

It was not at all easy to do. WebRTC has a very complicated handshake, and these GStreamer elements require a large number of special connections, but this gave the desired results.

Oh, and by the way, our experience is that openh264 works well for WebRTC traffic, and we ended up using it for many cases.

+2
source

Source: https://habr.com/ru/post/1269988/

More articles:


All Articles