HeartBeat Discovery Using WebCam?

Question

I am trying to create an application that can detect a heartbeat using your computer’s webcam. I have been working on the code since 2 weeks and developed this code, and here I am still

How it works? Illustrated below ...

• Face detection with opencv
• Getting a forehead image
• Apply a filter to convert it to grayscale [you can skip it]
• Search for average green pixel intensity per frame
• Saving averages to an array
• FFT application (I used a minimal library) Get heart rate from the FFT spectrum (I need help here)

Here I need help to extract the pulse from the FFT spectrum. Can someone help me. Here , it is a similar application developed in python, but I can not refuse this code, so I am developing the same in the process. Can anyone help me uncover the part of this python code where it extracts a heartbeat.

//---------import required ilbrary ----------- import gab.opencv.*; import processing.video.*; import java.awt.*; import java.util.*; import ddf.minim.analysis.*; import ddf.minim.*; //----------create objects--------------------------------- Capture video; // camera object OpenCV opencv; // opencv object Minim minim; FFT fft; //IIRFilter filt; //--------- Create ArrayList-------------------------------- ArrayList<Float> poop = new ArrayList(); float[] sample; int bufferSize = 128; int sampleRate = 512; int bandWidth = 20; int centerFreq = 80; //--------------------------------------------------- void setup() { size(640, 480); // size of the window minim = new Minim(this); fft = new FFT( bufferSize, sampleRate); video = new Capture(this, 640/2, 480/2); // initializing video object opencv = new OpenCV(this, 640/2, 480/2); // initializing opencv object opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE); // loading haar cscade file for face detection video.start(); // start video } void draw() { background(0); // image(video, 0, 0 ); // show video in the background opencv.loadImage(video); Rectangle[] faces = opencv.detect(); video.loadPixels(); //------------ Finding faces in the video ----------- float gavg = 0; for (int i = 0; i < faces.length; i++) { noFill(); stroke(#FFB700); // yellow rectangle rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height); // creating rectangle around the face (YELLOW) stroke(#0070FF); //blue rectangle rect(faces[i].x, faces[i].y, faces[i].width, faces[i].height-2*faces[i].height/3); // creating a blue rectangle around the forehead //-------------------- storing forehead white rectangle part into an image ------------------- stroke(0, 255, 255); rect(faces[i].x+faces[i].width/2-15, faces[i].y+15, 30, 15); PImage img = video.get(faces[i].x+faces[i].width/2-15, faces[i].y+15, 30, 15); // storing the forehead aera into a image img.loadPixels(); img.filter(GRAY); // converting capture image rgb to gray img.updatePixels(); int numPixels = img.width*img.height; for (int px = 0; px < numPixels; px++) { // For each pixel in the video frame... final color c = img.pixels[px]; final color luminG = c>>010 & 0xFF; final float luminRangeG = luminG/255.0; gavg = gavg + luminRangeG; } //-------------------------------------------------------- gavg = gavg/numPixels; if (poop.size()< bufferSize) { poop.add(gavg); } else poop.remove(0); } sample = new float[poop.size()]; for (int i=0;i<poop.size();i++) { Float f = (float) poop.get(i); sample[i] = f; } if (sample.length>=bufferSize) { //fft.window(FFT.NONE); fft.forward(sample, 0); // bpf = new BandPass(centerFreq, bandwidth, sampleRate); // in.addEffect(bpf); float bw = fft.getBandWidth(); // returns the width of each frequency band in the spectrum (in Hz). println(bw); // returns 21.5332031 Hz for spectrum [0] & [512] for (int i = 0; i < fft.specSize(); i++) { // println( " Freq" + max(sample)); stroke(0, 255, 0); float x = map(i, 0, fft.specSize(), 0, width); line( x, height, x, height - fft.getBand(i)*100); // text("FFT FREQ " + fft.getFreq(i), width/2-100, 10*(i+1)); // text("FFT BAND " + fft.getBand(i), width/2+100, 10*(i+1)); } } else { println(sample.length + " " + poop.size()); } } void captureEvent(Capture c) { c.read(); } 

Answer 1

FFT is applied in a window with 128 samples.

int bufferSize = 128;

During the drawing method, samples are stored in an array until the FFT buffer is full. Then the buffer is saved. To insert a new sample, the oldest of them is deleted. gavg is the medium gray color of the channel.

 gavg = gavg/numPixels; if (poop.size()< bufferSize) { poop.add(gavg); } else poop.remove(0); 

Copy feed to sample

 sample = new float[poop.size()]; for (int i=0;i < poop.size();i++) { Float f = (float) poop.get(i); sample[i] = f; } 

You can now apply the FFT to the Array sample.

  fft.forward(sample, 0); 

The code shows only the result of the spectrum. Heart rate should be calculated. For each band in fft you need to find the maximum, and this position is the heart rate.

  for(int i = 0; i < fft.specSize(); i++) { // draw the line for frequency band i, scaling it up a bit so we can see it heartBeatFrequency = max(heartBeatFrequency,fft.getBand(i)); } 

Then get the frequency band to know the frequency.

  float bw = fft.getBandWidth(); 

Frequency adjustment.

heartBeatFrequency = fft.getBandWidth () * heartBeatFrequency;