How to determine if it is normal to activate additional processing

Question

Motivation is difficult to explain, so I’ll give an example: suppose you get a large number of samples every second, and your task is to classify them.

Let's also say the following: you have two classifiers: Fast heuristic and heuristic. So let's say that for each sample you run heuristicFast (), and then if the result is close to indefinite (for example, [0.45,0.55] the range for the classifier, where 0 is class 1 and 1 is class 2) I run a more accurate heuristicSlow value.

Now the problem is that it is a real-time system, so I want to be sure that I do not overload the processors (I use threads), even when high call redirection to Fast heuristic returns the results in the [0,45,0,55] assortment.

What is the best way to do this?

My best idea is to have an input table for heuristicSlow and then not enter it if entrycount> number_of_cores / 2 ?

  std::atomic<int> entrycount(0); //... if (classificationNotClear(result_heuristic_fast) && (entrycount<kMaxConcurrantCalls)) { entrycount++; final_result=heuristicSlow(); entrycount--; } else final_result=result_heuristic_fast; //... 

Answer 1

An even more favorable solution:

Sort your fast heuristic results by uncertainty (i.e. abs(result-0.5) ) and perform a slow heuristic for as many cases as you have time left.

Answer 2

Since you are creating a real-time system, you have important information: the maximum allowable runtime for your classification and for both heuristics.

You could simply calculate the remaining time for a fully fast heuristic (total time minus the sample time with fast heuristic time) and determine how many slow heuristic applications fit at that time. Write this number in the counter and decrease.