Video Composition with Media Foundation

Question

I am trying to collect 2 videos into one (audio too) and save as a file. Say, show the source video in the left half of the final video and the source of the ocher on the right. Any encoding is fine. Not with DirectShow.

I suspected that IMFVideoMixerControl might be related to this, but unfortunately I'm too new to MF to determine the right step to achieve this. If you could give me guidance, or if any example is available on the Internet, please let me know. Thank you for your help.

Answer 1

IMFVideoMixerControl used to render video. You were on the right track looking at SourceReader.

Here is what I will do:

• Create an IMFSourceReader for each video source.
• Create an IMFSinkWriter and configure it with a frame width equal to the sum of the frame width of the source reader.
  • Use IMFSourceReader::GetCurrentMediaType to get the original media types.
  • Use MFGetAttributeSize with the GUID MF_MT_FRAME_SIZE to get the frame sizes for each type of source.
  • Create a media type for SinkWriter using MFCreateMediaType and use IMFMediaType::CopyAllItems to copy attributes from the source to flood media types.
  • Use the MFSetAttributeSize with the GUID MF_MT_FRAME_SIZE to set the dimensions of the oversized sink frames.
  • Use IMFSinkWriter::AddStream to create a video stream identical to the source type, except for the width attribute
• Call IMFSourceReader :: ReadSample for each source, giving you one IMFSample for each source.
• Highlight the new IMFSample by adding a new IMFMediaBuffer with increased frame width.
• Use MFCopyImage to copy each of the source buffers to the corresponding side of the selected media buffer.
• Use IMFSinkWriter::WriteSample to write your IMFSample to a raster file.

See this example for some basic processing of the SourceReader / SinkWriter, although this example uses a video capture source instead of the source. You can create a SourceReader file using MFCreateSourceReaderFromURL instead of MFCreateSourceReaderFromMediaSource .

Edit: I realized that you were asking about audio too. My answer only concerns the layout of video streams.