I am trying to implement real-time streaming using gstreamer as a server on top of pure RTP (without RTSP) using an sdp file with a server-to-client delay <500ms. Test server tested using
gst-launch-1.0 -v v4l2src ! video/x-raw,width=640,height=480,framerate=30/1 ! x264enc tune=zerolatency threads=0 bitrate=500 speed-preset=ultrafast ! rtph264pay pt=96 config-interval=1 ! udpsink host=XXXX port=X
If I try to get it from the gstreamer client using gst-launch, I do not get any delay (using two different devices on the network), I can also achieve the same using vlc as a client, reading the sdp file and using file-caching> = 1500 ms.
My problem is that I need an android client. I am trying to use LibVLC for Android, but with the same parameters I cannot achieve a delay of <2s.
LibVLC code for Android, where MediaPlayer and Media are part of org.videolan.libvlc:
ArrayList<String> options = new ArrayList<String>(); options.add("--file-caching=2000"); mLibVLC = new LibVLC(options); mLibVLC.setOnHardwareAccelerationError(this); mHolder.setKeepScreenOn(true); // Create media player mMediaPlayer = new MediaPlayer(mLibVLC); mMediaPlayer.setEventListener(mPlayerListener); mSurface = (SurfaceView) findViewById(R.id.surfaceView); mHolder = mSurface.getHolder(); // Set up video output final IVLCVout vout = mMediaPlayer.getVLCVout(); vout.setVideoView(mSurface); vout.setWindowSize(640, 480); vout.addCallback(this); vout.attachViews(); Media m = new Media(mLibVLC, media); mMediaPlayer.setMedia(m); mMediaPlayer.play();
I also tried other options: --clock-synchro --clock-jitter, etc ... At startup it seems that it takes about 2 seconds to buffer and these seconds delay the entire transfer. Is there a way to reduce the delay, as in the standard client, or if there is an alternative Android client?
DECISION:
Finally, I received it with a delay of ≈200 ms. I explored LibVLC and Media objects more, and I based that I can set local Media parameters and hardware acceleration. I set up network caching at a low level (unlike the VLC client client), clock synchronization, clock jitter, especially since I used the global option to scale Fast Bilinear algorithm, which is enough for my activity. Below updated code:
ArrayList<String> options = new ArrayList<String>(); options.add("--aout=none"); options.add("--swscale-mode=0"); mLibVLC = new LibVLC(options); mLibVLC.setOnHardwareAccelerationError(this); mHolder.setKeepScreenOn(true); // Create media player mMediaPlayer = new MediaPlayer(mLibVLC); mMediaPlayer.setEventListener(mPlayerListener); // Set up video output final IVLCVout vout = mMediaPlayer.getVLCVout(); vout.setVideoView(mSurface); vout.setWindowSize(640, 480); vout.addCallback(this); vout.attachViews(); Media m = new Media(mLibVLC, media); m.setHWDecoderEnabled(true, false); m.addOption(":network-caching=150"); m.addOption(":clock-jitter=0"); m.addOption(":clock-synchro=0"); mMediaPlayer.setMedia(m); mMediaPlayer.play();
Turning on hardware acceleration on the media automatically sets file caching and network caching up to 1500 ms, so I overwrote the network caching option, in this case file caching is not used. Further research could be done using the clock-jitter option for better optimization.