Real-time music recording

I am trying to introduce a system that can create a sheet of music for a piano piece when and when it is being played, that is, there should be real-time transcription, not a recording of the recorded fragment.

p>

So far I have tested wav files using FFt on MATLAB , but when it comes to real time, the system should be very very fast .

Any suggestions on which approach I could use , board type (I was thinking about using Arduino ). I could use it because I would like to migrate my MATLAB code rather than convert it to C.

+6
source share
1 answer

Any suggestions on which approach I could use, board type (I was thinking about using Arduino). I could use it because I would like to pass MATLAB code directly, and not convert it to C.

You may have different approaches to your project that I can think of:

  • First, there are some pianos that have an electronic device that can output reproduced notes through the midi interface with speed and position. This is the most accurate and fast solution for tracking recorded notes, although not a lot of DIY. On your computer, you only need the midi interface (which can be easily done using arduino) and have software that converts MIDI notes to a score;
  • You can then imitate this behavior by swiping the piano keyboard on sensors that can measure speed and which note to play. There are many strategies that you can use (from using lasers from mechanics to pressure switches ...), it all depends on the investments you are ready to make; There arduino (e.g. mega or due) may be a good solution.
  • But since you were talking about FFT, I assume that you want to record the sound made by the piano and discover the keys. There you got acquainted with modern knowledge in the field of musical research, the problem that you will encounter is the multiple detection of f0 in polyphonic music , although there are strategies specific to the piano , this is still a very active area of ​​research.
  • Suppose you are doing monophonic detection for transcription. The problem will be that the A / D converter included in the Arduino just sucks, it has a 10bit definition that is too low to find anything useful. With Arduino DUE, you can have a few more definitions since it is equipped with a 16-channel 12-bit 1 Mbit / s ADC. What is still not very good for good transcription. Thus, you will either have to create an external ADC like Linear LTC2383-16 , which has 16 bits 1 Mbit / s and has an SPI interface, the minimum start playing. For this, you will need to find the right microphone and create good electronics between the ADC and the microphone (which I know little about).
  • Finally, would it not be easier to take a small computer (for example, Beaglebone or RaspberryPI 5 ), plug in a really good USB ADC / sound card and use this board for transcription? You can also run your software on a cloud server (like Google appengine or AWS) that has enough processor to make your Matlab work happily.

To make a good time and built-in transcription of music, the first solution is the best solution (or the second if you really want to go DIY). If you really want to go through the air interfaces, I think the fifth solution is a good solution.

And finally, about the code, I would advise you to convert your code from Matlab to Python (if you do not want to write C / C ++ code), it will work much faster. That way, you can understand how bad Matlab is when I worked in IRCAM, I wrote a tool that converted Matlab source code to C ++ source code, and once the compiled result was 40 times faster.

NTN

+1
source

Source: https://habr.com/ru/post/948114/


All Articles