I am currently creating an application that is an image filter that should run in real time. It filters, converts the image taken from the camera, and displays the converted image, as the camera preview does.
The problem I am facing with this is that performance is very important for this application, since each captured pixel needs to be converted, this gives approximately 640x480 pixels x3 (color options, R, G, B) x about 30- 50 operations per color measurement, then converting it to RGB and creating a new image.
First, when I tested the functional requirements of the algorithm in Matlab, it took about 5-15 seconds to convert the 600x600 image to a quadcore processor. Then I reviewed the algorithm on the target platform, which is C ++ for Windows mobile devices, and now I get about 0.2-0.7 frames per second when processing the same image.
There are already many optimizations in the C ++ version, such as pre-caching and pre-calculating the values of each color and storing pre-calculated values in trees, reducing distances between data in memory, performing operations with bit operators, rather than adding and multiplying, when applicable but still the performance was not satisfactory.
I tried to find out the bottleneck of the whole algorithm and started with a simple empty TransformFilter, which had a video camera at one end of the CaptureGraph, and on the other hand a window handle and visualization tools that should show the camera’s preview without any action. This turned out to be the biggest bottleneck!
A clean preview in a window app on Windows Mobile 6.5 gave me an average of 1-2 frames per second, and I'm sure this is not a matter of testing on a slow phone. I am testing it on HTC Diamond 2. It has ~ 530 MHz, and the camera application shows smooth preview at about 15-20 FPS, even if any effect is applied (for example, BW, Sofia), the preview performance is very high.
What am I doing wrong?
, ?
- "" Windows Mobile, ?
.