Dynamically change panning of an audio stream in Swift using an HLS source (m3u8)

Question

Over the past few days, I have been trying to find a way to dynamically change (during playback) the value of the "panning" of the stereo audio stream in the video stream via HLS. The end result I want is to disable the right channel or left channel during video playback. I thought it would be easy.

I tried a couple of options, none of which brought me joy and would really appreciate any pointers that could solve my problems. I will tell you what I tried - I hope someone can help.

Trying first I tried using Apple MTAudioProcessingTap. I read in various comments that this is not possible to use over HLS, but I thought it was worth going for it. Here is an example of my code:

if let playerItem = player?.currentItem, let audioTrack = getAudioTrack(playerItem: playerItem) {
    let inputParams = AVMutableAudioMixInputParameters(track: audioTrack)
    let callbacks = MTAudioProcessingTapCallbacks(version: kMTAudioProcessingTapCallbacksVersion_0,
                                                  clientInfo: UnsafeMutableRawPointer(Unmanaged.passUnretained(self).toOpaque()),
                                                  init: tapInit,
                                                  finalize: tapFinalize,
                                                  prepare: tapPrepare,
                                                  unprepare: tapUnprepare,
                                                  process: tapProcess)

     let err = MTAudioProcessingTapCreate(kCFAllocatorDefault, &callbacks!, kMTAudioProcessingTapCreationFlag_PostEffects, &tap)
     assert(tap != nil)
     inputParams.audioTapProcessor = tap!.takeunRetainedValue()

     let audioMix = AVMutableAudioMix()
     audioMix.inputParameters = [inputParams]
     playerItem.audioMix = audioMix
}

I noticed that calling callInit calls the call, however, none of the other callbacks are called before tapFinalize. When downloading the file locally, all the callbacks fall as I expected, including the process callback. As a result of this, I did not try to actually change the value of the pan.

Trying second I tried using the CoreAudio API. Despite the fact that everything was configured as I expected, I do not see any effect on the audio output when downloading content via HLS or locally. Here is my setup:

var processingGraph: AUGraph? = nil
var status = OSStatus(noErr)
var mixerUnit: AudioUnit? = nil
var outputUnit: AudioUnit? = nil

status = NewAUGraph(&processingGraph)

var mixerNode = AUNode()
var mixerUnitDescription = AudioComponentDescription(
    componentType:         OSType(kAudioUnitType_Mixer),
    componentSubType:      OSType(kAudioUnitSubType_MultiChannelMixer),
    componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
    componentFlags:        0,
    componentFlagsMask:    0)

var outputNode = AUNode()
var outputUnitDescription = AudioComponentDescription(
    componentType:         OSType(kAudioUnitType_Output),
    componentSubType:      OSType(kAudioUnitSubType_RemoteIO),
    componentManufacturer: OSType(kAudioUnitManufacturer_Apple),
    componentFlags:        0,
    componentFlagsMask:    0)

status = AUGraphAddNode(processingGraph!, &outputUnitDescription, &outputNode)
status = AUGraphAddNode(processingGraph!, &mixerUnitDescription, &mixerNode)

let outputElement = AudioUnitElement(0)
let mixerElement = AudioUnitElement(0)
status = AUGraphConnectNodeInput(processingGraph!, mixerNode, mixerElement, outputNode, outputElement)

status = AUGraphNodeInfo(processingGraph!, mixerNode, nil, &mixerUnit)
status = AUGraphNodeInfo(processingGraph!, outputNode, nil, &outputUnit)

status = AUGraphOpen(processingGraph!)

var outIsInited: DarwinBoolean = false
status = AUGraphIsInitialized(processingGraph!, &outIsInited)
if outIsInited == false {
    print("Need to init graph")
    status = AUGraphInitialize(processingGraph!)
}

var isRunning: DarwinBoolean = false
status = AUGraphIsRunning(processingGraph!, &isRunning)
if isRunning == false {
    print("Need to start graph")
    status = AUGraphStart(processingGraph!)
}

Then I set the pan value like this:

status = AudioUnitSetParameter(mixerUnit!, kMultiChannelMixerParam_Pan, kAudioUnitScope_Input, 0, panValue, 0)

, , , , .

, Superpowered SDK, , , SuperpoweredStereoMixer, , :

( ), ( )

, - , HLS.