How to test methods that may not always give the correct answer

Question

Imagine you have a list of suppliers with internal controls. Now imagine that you want to map unstructured lines to this list. Most of them will be easy to match, but some of them may be almost impossible. The algorithm assigns confidence to each match, but a person needs to confirm all matches made.

How can this module be checked by the module? The only idea I have had so far is to select pairs agreed by people, and make sure that the algorithm is able to successfully match those that skip strings that I could not reasonably expect from our algorithm. Is there a better way?

Answer 1

I would try some “canonical” pairs, the pairs “must match” and “should not match,” and only check if the trust is above (or below) the given threshold.

perhaps you can also perform some order checks, for example, "no pair should have more confidence than one that matches the exact pair of matches," or "a pair that matches all consonants should be> = only vowel."

Answer 2

You can also check if you are confident enough that your algorithm will fail. Thus, you can see if there is a threshold by which you can trust your algorithm.

Answer 3

, , , , .

, .

Answer 4

, , , ; , , , . , , , ; , . , , .

Answer 5

. ( ), , , . , , X% .

.

Answer 6

, : (.. , , , ).

1 - . 2 (.. ). ​​

Answer 7

, , - , , , .

Answer 8

, , , , , ( ).

" ". , , . , , .

- - , , , , .

Samples that have clearly correct answers (exact matches, zero matches, corner cases with a high value) should be stored in a separate test.