I am currently working on a small project where I create a CubeMap using WebGL and then apply some sounds using the "Web Audio API " web audio API
Edit: Real-time example
Since the project is very large, I'm just trying to explain what I'm looking for. When I upload an audio file, the sounds are visualized (looks like a cube). The position of the audio listener is ALWAYS in the position 0,0,0. What I have done so far is that I created a βCameraβ ( gl math library ) with lookAt and perspective and when I turn the camera away from the sound-emitting cube, the reproduced sound should sound different.
How am i doing this?
Each frame I set the orientation of the PannerNode ( panner node to set the orientation ) on the upVector camera. Here is the update -Method frame (for sound):
update(camera) { this._upVec = vec3.copy(this._upVec, camera.upVector); //vec3.transformMat4(this._upVec, this._upVec, camera.vpMatrix); //vec3.normalize(this._upVec, this._upVec); this._sound.panner.setOrientation(this._upVec[0], this._upVec[1], this._upVec[2]); }
And here is the updateViewProjectionMethod -Methof from my Camera class, where I update the listener Orientation:
updateViewProjMatrix() { let gl = Core.mGetGL(); this._frontVector[0] = Math.cos(this._yaw) * Math.cos(this._pitch); this._frontVector[1] = Math.sin(this._pitch); this._frontVector[2] = Math.sin(this._yaw) * Math.cos(this._pitch); vec3.normalize(this._lookAtVector, this._frontVector); vec3.add(this._lookAtVector, this._lookAtVector, this._positionVector); mat4.lookAt(this._viewMatrix, this._positionVector, this._lookAtVector, this._upVector); mat4.perspective(this._projMatrix, this._fov * Math.PI / 180, gl.canvas.clientWidth / gl.canvas.clientHeight, this._nearPlane, this._farPlane); mat4.multiply(this._vpMatrix, this._projMatrix, this._viewMatrix); Core.getAudioContext().listener.setOrientation(this._lookAtVector[0], this._lookAtVector[1], this._lookAtVector[2], 0, 1, 0); }
Is it correct? I hear that the sound changes if I turn the camera, but I'm not sure. And do I need to multiply the result upVector by the current viewProjectionMatrix ?