Speex split audio data - WebAudio - VOIP

Question

Im running a small application that encodes and decodes a sound array using speex codec in javascript: https://github.com/dbieber/audiorecorder

with a small array filled with sin signal

for(var i=0;i<16384;i++) data.push(Math.sin(i/10)); 

it works. But I want to create a VOIP application and have more than one array. Therefore, if I split my array into 2 parts, then encode> decode> merge, it will not be the same as before.

Take a look at this:

script: http://jsfiddle.net/exh63zqL/

Both buttons must have the same audio output.

How can I get the same result in both directions? Is their special mode in speex.js for splitting audio data?

Answer 1

Please note that Speex is a lossy codec . Thus, by definition, it cannot give the same result as the encoded buffer. In addition, it is designed to encode voice. Thus, a range of 1-2 kHz will be most effective, since it expects a certain waveform. In some ways, it can be compared with JPEG technology for bitmaps.

I slightly modified your jsfiddle example so that you can play with different parameters and compare the results. Simply providing a simple sinusoid with an unknown frequency is not the proper way to test a codec. However, in this example you can see different effects on the original signal at different frequencies.

 buffer1.push(Math.sin(2*Math.PI*i*frequency/sampleRate)); 

I think you should create an example with a recorded voice and compare the results in this case. That would be more correct.

In general, to get detailed information, you will need to study the processing of a digital signal. I can’t even provide the correct connection, because it is a whole science, and it is mathematically intense. (the only correct reading book I know is in Russian ). If someone here with a strong mathematical background can share the relevant literature for this case, I would appreciate it.

EDITOR: as Kuroi Neko mentioned, there are problems with buffer boundaries. And it seems that the state of the decoder cannot be saved, as mentioned in this post, because the library used does not support it. If you look at the source code, you will see that they use a third-party speex codec and do not provide full access to its functions. I think the best approach would be to find a decent library for speex that supports state recovery like this

Answer 2

Speex is a lossy codec, so the output is only an approximation to the original sine wave.

Your sine wave frequency is around 7 kHz, which is close to the top codec with a frequency of 8 kHz and, as such, is more likely to be changed.

What outputs of the codec look like a dirach pulse comb, which will sound like your original sine wave, as heard through the phone, which is certainly different from the original.

See this script where you can listen to what the codec is doing from your original sine waves, be split in half or not.

 //Generate a continus sinus in 2 arrays var len = 16384; var buffer1 = []; var buffer2 = []; var buffer = []; for(var i=0;i<len;i++){ buffer.push(Math.sin(i/10)); if(i < len/2) buffer1.push(Math.sin(i/10)); else buffer2.push(Math.sin(i/10)); } //Encode and decode both arrays seperatly var en = Codec.encode(buffer1); var dec1 = Codec.decode(en); var en = Codec.encode(buffer2); var dec2 = Codec.decode(en); //Merge the arrays to 1 output array var merge = []; for(var i in dec1) merge.push(dec1[i]); for(var i in dec2) merge.push(dec2[i]); //encode and decode the whole array var en = Codec.encode(buffer); var dec = Codec.decode(en); //----------------- //Down under is only for playing the 2 different arrays //------------------- var audioCtx = new window.AudioContext || new window.webkitAudioContext; function play (sound) { var audioBuffer = audioCtx.createBuffer(1, sound.length, 44100); var bufferData = audioBuffer.getChannelData(0); bufferData.set(sound); var source = audioCtx.createBufferSource(); source.buffer = audioBuffer; source.connect(audioCtx.destination); source.start(); } $("#o").click(function() { play(dec); }); $("#c1").click(function() { play(dec1); }); $("#c2").click(function() { play(dec2); }); $("#m").click(function() { play(merge); }); 

If you combine the two outputs of a signal decoder, you will hear an additional click due to a sharp transition from one signal to another, which sounds mostly like switching a relay.
To avoid this, you will have to smooth the values ​​around the merge point of the two buffers.