I have a multi-threaded process. Each thread is associated with a CPU (performs calculations) and also uses a lot of memory. The process begins with 100% processor utilization according to the resource monitor, but after a few hours the processor utilization begins to slowly deteriorate. After 24 hours, it is 90-95% and falls.
The question is what should I look for and what are the most famous methods I can use to debug this?
Additional Information:
I have enough RAM - most of it is not used at any time. According to perfmon, memory does not grow (so I don’t think it is leaking). The code is a combination of .Net and native C ++ with some data marshaling back and forth. I saw this on several different machines (servers with 24 logical cores). One thing I saw in perfmon is the Changed Page List Indicator, which increases over time as CPU utilization worsens.
Edit 1
One of the third-party libraries that is used is openfst. This seems to be very related to the misuse of this library. In particular, I noticed that I have the following warnings: warning LNK4087: the CONSTANT keyword is out of date; use DATA
Edit 2
Since the question is closed and has not been reopened, I will write my conclusions and how the problem was solved in the body of the question (sorry) for future users. It turns out that there is an openfst.def file that defines all the openfst FLAGS_ * characters that will be used when using / dll applications. I had to fix this in order to use the keyword "DATA" instead of "CONSTANT" (CONSTANT is deprecated because it is risky - more info: https://msdn.microsoft.com/en-us/library/aa271769(v=vs .60) .aspx) After that, there was no longer a deterioration in CPU usage. The “modified lists” indicator does not increase. I suspect this is due to the default values of FLAGS (in particular, garbage collection flags - FLAGS_fst_default_cache_gc), which were not deterministic due to the incorrect use of the CONSTANT keyword in the openfst.def file.
Conclusion Understand your warnings! Eliminate as many as you can! Thanks.
source
share