WebGL Audio Space

I am currently working on a small project where I create a CubeMap using WebGL and then apply some sounds using the "Web Audio API " web audio API

Edit: Real-time example

Since the project is very large, I'm just trying to explain what I'm looking for. When I upload an audio file, the sounds are visualized (looks like a cube). The position of the audio listener is ALWAYS in the position 0,0,0. What I have done so far is that I created a β€œCamera” ( gl math library ) with lookAt and perspective and when I turn the camera away from the sound-emitting cube, the reproduced sound should sound different.

How am i doing this?

Each frame I set the orientation of the PannerNode ( panner node to set the orientation ) on the upVector camera. Here is the update -Method frame (for sound):

  update(camera) { this._upVec = vec3.copy(this._upVec, camera.upVector); //vec3.transformMat4(this._upVec, this._upVec, camera.vpMatrix); //vec3.normalize(this._upVec, this._upVec); this._sound.panner.setOrientation(this._upVec[0], this._upVec[1], this._upVec[2]); } 

And here is the updateViewProjectionMethod -Methof from my Camera class, where I update the listener Orientation:

  updateViewProjMatrix() { let gl = Core.mGetGL(); this._frontVector[0] = Math.cos(this._yaw) * Math.cos(this._pitch); this._frontVector[1] = Math.sin(this._pitch); this._frontVector[2] = Math.sin(this._yaw) * Math.cos(this._pitch); vec3.normalize(this._lookAtVector, this._frontVector); vec3.add(this._lookAtVector, this._lookAtVector, this._positionVector); mat4.lookAt(this._viewMatrix, this._positionVector, this._lookAtVector, this._upVector); mat4.perspective(this._projMatrix, this._fov * Math.PI / 180, gl.canvas.clientWidth / gl.canvas.clientHeight, this._nearPlane, this._farPlane); mat4.multiply(this._vpMatrix, this._projMatrix, this._viewMatrix); Core.getAudioContext().listener.setOrientation(this._lookAtVector[0], this._lookAtVector[1], this._lookAtVector[2], 0, 1, 0); } 

Is it correct? I hear that the sound changes if I turn the camera, but I'm not sure. And do I need to multiply the result upVector by the current viewProjectionMatrix ?

+5
source share

Source: https://habr.com/ru/post/1263058/


All Articles