Crawd real-time video object recognition and collision detection library or recommendation tool

Question

We need to be able to analyze HD video scenes of the crowd from above (I think the train station) and be able to detect all moving objects and perform collision detection. The virtual object will be effectively superimposed on the image and should correspond to real objects. This should happen in real time.

Ideally, I am looking for a library that can be used with C # for this. This is a way out of my comfort zone, so I'm just looking for some recommendations as to whether this is a difficult or difficult problem and what tools will work. Is OpenCV suitable, for example?

If there are performance limitations for how many objects can be processed, what would it be?

Answer 1

Short answer: Not really

Long answer: There are some libraries that do what you want to do, but all of them, as I know, are commercial things, they are expensive, and they are used in security applications. It is highly unlikely that you will get the SDK at a reasonable price and will be flexible enough to allow you to play with it. (Most likely, it will only output the number of people or something like that)

If you still want to do this job, don't be disappointed: OpenCV is a great tool to help you, but it definitely eats up a significant portion of your time. Google crowd counting/tracking/detection in video , find good paper and implement it. But keep in mind that this may require a thorough knowledge of mathematics and image processing.

Processing Speed ​​Tip: It will not work in real time unless you write a parallelizable algorithm and run it on multiple GPUs / processors, and carefully configure it. However, with a good implementation, you should have a reasonable speed.

And the last tip: if you really want it, working in C ++ will be the least difficult part. It is much better to use the official opencv (C ++) than the C # shell (the best documents, modern functions, all functions for you). I don’t know of any serious project using .NET wrappers - only students play around.