A smart strategy for expected and expected unconditional module testing

Question

I would like some ideas on how I should test some objects that might be blocked to wait for another member. The specific module under test is the channel between the participants, the participants themselves are prototype lamps for testing purposes.

It would be nice to confirm that the participants are deadlocked when they are expected, but for me this is not very important, because what happens after the deadlock can be reasonably described as undefined.

More important would be to make sure that certain interactions of the participants were not at an impasse.

In any case, I'm not sure what the optimal testing strategy should be. My real concept is to run a test running thread for each participant, sleeping a bit, and then find out if the child threads have returned. In case they didn’t return on time, suppose that they have reached a dead end and safely terminate the threads and the test fails (or succeeds if a dead end is expected).

This seems a bit plausible, as there may be all sorts of reasons (as if unlikely) that the thread may take longer than expected. Are there any other good ways to solve this problem?

EDIT: I'm sure testing will be nice, but I don't think I need this. I think of three levels of validation.

• "Actual behavior has proven to be consistent with expected behavior." "Dead end can't happen"
• “Actual behavior consistent with expected behavior” deadlock not seen in N tests
• "Actual behavior is consistent with expected behavior" N tests completed in expected time

The first, of course, is a valuable test for passing, but ShiDoiSi's answer suggests this is impractical. The second is much weaker than the first, but still heavy; How can you establish that a network of processes has actually reached an impasse? I'm not sure it's easier to prove than the first (maybe a lot harder)

The latter is more like what I mean.

Answer 1

The only way to reliably test deadlocks is to force a subsystem lock to detect and report them. The last time I had to do this, we built a debug version that recorded threads that are blocked and checked for possible deadlocks during each call-call. It can be a difficult operation in a system with a large number of locks, but we found that it was so valuable that we reorganized the subsystem so that we could turn it on and off using the switch at runtime even in production assemblies.

Answer 2

The academic community is likely to tell you (actually it tells you right now;) that you should make a true abstraction into some kind of so-called model verification system ( CSP, pi-calculus ). Then it will imitate abstract executions (exhaustive search through all possible transitions of the scheduler). Of course, the trick is to make sure that the abstraction is really true. You no longer check the actual source of your program, but the source in some other language.

Otherwise, some kind of hard approach comes to mind, for example, using the Java Path Finder / Explorer (which does something very similar) for a particular language. Similar research prototypes exist for C and Intel and other companies also in this business with specialized tools.

You study one of the hot topics in research in the field of computer science, and for non-trivial / real systems, neither exhaustive testing nor formal verification can be easily applied to real code.

A valuable approach may be to measure your code so that it actually detects a dead end and potentially tries to recover. To detect deadlocks, the FreeBSD kernel uses a set of C-macros that track the use of locks and report potential violations through the witness(4) mechanism. But then again, errors that are rare will rarely occur.

(Disclaimer: I do not participate in any commercial tools linked above). I just added them to make you feel the difficulty of the problem you are facing.)

Answer 3

For testing, if there is no deadlock, you can use the equivalent of the NUnit TimeoutAttribute, which interrupts and fails checking if the execution time exceeds the upper limit. You can come with a good timeout value, for example, if the test does not complete within 30 s, something is wrong.

I am not sure (or I have not come across a situation) about the claim that a dead end has occurred. Dead ends are usually undesirable. I don’t understand how to write a unit test that fails if the test blocks are unit tests, as a rule, should be fast and non-blocking.

Answer 4

Since you have already done enough abstraction to mock the participants, why not distract them further and abstract your synchronization of flows (mutex, semaphore, and much more)?

When you think about what constitutes a dead end, you can use specialized synchronization flow in the dead end in your tests. By "deadlock-aware" I do not mean that he should detect deadlocks by brute force, using timeouts, etc., but be aware of situations that lead to deadlocks through flags, counters, etc. It can detect deadlocks, while not necessarily providing the expected thread synchronization functionality. Basically I say, using synchronized thread synchronization for your tests ...

This is too abstract and easier said than done. And I do not claim that I did it successfully. I can just be stupid here. But perhaps if you could provide only one (incomplete) test, the problem could be attacked more specifically.