You need at least four to get anything at all, but I found 7 to 10 to be a good number in practice, provided they are selected βmanuallyβ. Using some kind of function detector, you may need ten or 20 times to find a suitable subset with multiple outliers.
In fact, it depends on the distribution of points - if you have 10 linear points, you will not get a happy result, for example. However, for a slightly distorted rectangular display, all four corners are all you need.
source share