How to write a web music visualizer?

Question

I am trying to find the best approach for creating a music visualizer for working in a browser over the Internet. Unity is an option, but I will need to create a custom sound import / analysis module to get the end-user sound output. Quartz does what I need, but only works on Mac / Safari. WebGL seems not ready. Raphael is mostly 2D, and the question still remains how to make the user sound ... any ideas? Has anyone done this before?

Answer 1

Since WebGL is “not ready,” I assume that you mean intrusion (it is only supported in WebKit and Firefox at the moment).

In addition, equalizers are certainly possible using HTML5 audio and WebGL. A guy named David Humphrey wrote about creating different music visualizers using WebGL and was able to create really impressive ones. Here are some video images (click to view):

Answer 2

Making something sound responsive is pretty simple. Here is an open source site with lots of sound reactive examples .

As for this, you mainly use the Web Audio API to stream music and use your AnalyserNode to receive audio data.

"use strict"; // make a Web Audio Context var context = new AudioContext(); var analyser = context.createAnalyser(); // Make a buffer to receive the audio data var numPoints = analyser.frequencyBinCount; var audioDataArray = new Uint8Array(numPoints); var ctx = document.querySelector("canvas").getContext("2d"); function render() { ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height); // get the current audio data analyser.getByteFrequencyData(audioDataArray); const width = ctx.canvas.width; const height = ctx.canvas.height; const size = 5; // draw a point every size pixels for (let x = 0; x < width; x += size) { // compute the audio data for this point const ndx = x * numPoints / width | 0; // get the audio data and make it go from 0 to 1 const audioValue = audioDataArray[ndx] / 255; // draw a rect size by size big const y = audioValue * height; ctx.fillRect(x, y, size, size); } requestAnimationFrame(render); } requestAnimationFrame(render); // Make a audio node var audio = new Audio(); audio.loop = true; audio.autoplay = true; // this line is only needed if the music you are trying to play is on a // different server than the page trying to play it. // It asks the server for permission to use the music. If the server says "no" // then you will not be able to play the music // Note if you are using music from the same domain // **YOU MUST REMOVE THIS LINE** or your server must give permission. audio.crossOrigin = "anonymous"; // call `handleCanplay` when it music can be played audio.addEventListener('canplay', handleCanplay); audio.src = "https://twgljs.org/examples/sounds/DOCTOR%20VOX%20-%20Level%20Up.mp3"; audio.load(); function handleCanplay() { // connect the audio element to the analyser node and the analyser node // to the main Web Audio context const source = context.createMediaElementSource(audio); source.connect(analyser); analyser.connect(context.destination); } 

 canvas { border: 1px solid black; display: block; } 

 <canvas></canvas> 

Then you just need to do something creative.

pay attention to some problems that you are likely to encounter.

• At the moment (2017/1/3), neither Android Chrome support nor iOS Safari support analyze streaming audio data. Instead, you need to download the whole song. Here is a library that is trying to get a little distracted

• On a mobile device, you cannot automatically play audio. You must trigger the sound inside the input event based on user input, such as 'click' or 'touchstart' .

• As indicated in the example, you can only analyze audio if the source is either from the same domain, or you are requesting CORS permission and the server gives permission. AFAIK only Soundcloud gives permission, and this is song-based. It depends on the individual settings of the performers, regardless of whether sound analysis is allowed for a particular song.

  To try to explain this part

  By default, you have permission to access all data from the same domain, but without the permission of other domains.

  When you add

   audio.crossOrigin = "anonymous"; 

  It basically says, "ask the server for permission for an anonymous user." The server can give permission or not. This is up to the server. This includes a request even to a server in the same domain, which means that if you are going to request a song in the same domain, you must either (a) delete the line above, or (b) configure the server to grant CORS permission. Most servers do not give CORS permission by default, so if you add this line, even if the server is the same domain, if it does not give CORS permission, then an attempt to analyze the sound will fail.

---

Music: DOCTOR VOX - Level Up

Answer 3

I used SoundManager2 to extract waveform data from an mp3 file. This feature requires Flash 9, so this may not be the best approach.

My demo with HMTL5 Canvas: http://www.momentumracer.com/electriccanvas/

and WebGL: http://www.momentumracer.com/electricwebgl/

Sources: https://github.com/pepez/Electric-Canvas

Answer 4

Depending on the complexity, it may be interesting for you to try processing (http://www.processing.org), it has really easy tools for creating web applications, and it has tools for getting FFT and waveforms of an audio file.