Convert Image to C #

Question

Edit: SOLVED! See my answer below for more details. I could not find the answer to the original question, but found an alternative solution

This question can be asked elsewhere, but I have been looking for days and cannot find anything that will help.

Question: I need to convert "Stream" to "image (bgr, byte)" at a time, is there a way / command to convert directly from System.Drawing.Image.FromStream to Emgu.CV.Image (Bgr, Byte) without conversion from stream to image to bitmap (bgr, byte)?

Info: I am coding in C # in Visual Studio 2010 as part of my dissertation project. I take a stream of images from an IP camera on the network and use many algorithms to detect faces / extracts of facial functions and face recognition. On my laptop, the local camera I can achieve FPS about 25 ~ (give or take), including algorithms, because I do not need to convert the image. For the IP camera stream, I need to convert it many times to achieve the desired format, and the result is about 5-8 frames per second.

(I know that my current method is extremely inefficient, so here I am, I actually convert the image 5 times (even gray scale), in fact, only half the memory of my processors (i7, 8 GB RAM)). This should be an image (bgr, byte), since this is the only format that algorithms will work with.

The code I use to get the image:

//headers using System.IO using System.Threading; using System.Net; //request a connection req = (HttpWebRequest)HttpWebRequest.Create(cameraUrl); //gives chance for timeout for errors to occur or loss of connection req.AllowWriteStreamBuffering = true; req.Timeout = 20000; //retrieve response (if successfull) res = req.GetResponse(); //image returned stream = res.GetResponseStream(); 

I have many things related to managing connections, data, security, etc., which I have reduced to the above code. My current code is to hide the image until the desired result:

 //Convert stream to image then to bitmap Bitmap bmpImage = new Bitmap(System.Drawing.Image.FromStream(stream)); //Convert to emgu image (desired goal) currentFrame = new Emgu.CV.Image<Bgr, Byte>(bmpImage); //gray scale for other uses gray = currentFrame.Convert<Gray, Byte>(); 

I understand that there is a way to save the image locally temporarily, but I will need to avoid this for security reasons. I am looking more for direct conversion to help conserve computing power. Am I missing something? All help is appreciated.

Thank you for reading. (I will update this if someone asks for more details) -Dave

Answer 1

I believe that I have found the answer to my problem. I tried to use Vano Maysuradze's idea of ​​processing in memory, which improved fps tiny margin (not immediately noticeable without testing). And also thanks to Plinth Answer: I have an understanding of multithreading, and I can optimize it as I move, because I can divide the algorithms into parallel work.

I believe my reason is network speed! not the actual delay of the algorithm. As Vano noted with a stopwatch, in order to find speed, the algorithms did not actually consume this. Thus, with and without algorithms, the speed is approximately the same if I optimize the use of threads so that the next frame is assembled, since the previous one completes the processing.

I did some testing on some of the physical Cisco routers and got the same result if I tinkered a bit slower with the clock speed and bandwidth, which was noticeable. Therefore, I need to find a way to quickly get frames over networks, many thanks to everyone who answered, who helped me better understand!

Output:

• Multithreading for optimization
• In-memory processing instead of constant conversion
• Improved network solutions (higher bandwidth and speed)

Edit: code for image retrieval and processing in memory for those who find it to search

 public void getFrames(object sender, EventArgs e) {//Gets a frame from the IP cam //Replace "IPADDRESS", "USERNAME", "PASSWORD" //with respective data for your camera string sourceURL = "http://IPADDRESS/snapshot.cgi?user=USERNAME&pwd=PASSWORD"; //used to store the image retrieved in memory byte[] buffer = new byte[640 * 480]; int read, total = 0; //Send a request to the peripheral via HTTP HttpWebRequest req = (HttpWebRequest)WebRequest.Create(sourceURL); WebResponse resp = req.GetResponse(); //Get the image capture after recieving a request //Note: just a screenshot not a steady stream Stream stream = resp.GetResponseStream(); while ((read = stream.Read(buffer, total, 1000)) != 0) { total += read; }//While End //Convert memory (byte) to bitmap and store in a picturebox pictureBox1.Image = (Bitmap)Bitmap.FromStream(new MemoryStream(buffer, 0, total)); }//getFrames End private void button1_Click(object sender, EventArgs e) {//Trigger an event to start running the function when possible Application.Idle += new EventHandler(getFrames); }//Button1_Click End 

Answer 2

You have a couple of potential bottlenecks, not least from the fact that you are probably jpeg, decode the stream into an image, and then convert it to a bitmap and then to an openCV image.

One way to get around this completely. This will be due to an attempt to use libjpeg directly. There is a free port for this here in C #, and IIRC you can connect to it to receive a call on each line to fill the buffer.

The downside is that you decrypt the JPEG data in a managed code that will run at least 1.5X slower than the C equivalent, although, to be honest, I would expect the network speed to change dramatically.

OpenCV should be able to directly read jpeg images (want to guess what they use under the hood? Survey says: libjpeg), which means you can buffer the entire stream and pass it to OpenCV and completely bypass the .NET layer.

Answer 3

You can save several images in memory (buffer), and then start processing from the buffer.

Something like that:

 //Convert stream to image then to bitmap Bitmap bmpImage = new Bitmap(System.Drawing.Image.FromStream(stream)); //Convert to emgu image (desired goal) currentFrame = new Emgu.CV.Image<Bgr, Byte>(bmpImage); //gray scale for later use gray = currentFrame.Convert<Gray, Byte>(); SaveToBuffer(gray); Queue<Emgu.CV.Image<Gray, Byte>> buffer = new Queue<Emgu.CV.Image<Gray, Byte>>(); bool canProcess = false; // ... private void SaveToBuffer(Emgu.CV.Image<Gray, Byte> img) { buffer.Enqueue(img); canProcess = buffer.Count > 100; } private void Process() { if(canProcess) { buffer.Dequeue(); // Processing logic goes here... } else { // Buffer is still loading... } } 

But note that you need enough RAM to store the images in memory, and you must also adjust the buffer size so that the meat meets your requirements.