Questions about some data mining algorithms

Question

I recently studied the k-nearest neighbor and decision trees, and I’m very interested to know about the difference between them, i.e. for the task, for example, to separate the objective function "return 1 if x2> x1, return 0 otherwise", then selecting the Nearest neighbor will be good here, since the decision tree will cause too many splits. Therefore, I am simply considering the question of when a scaled decision tree would be more suitable than a k-nearest neighbor?

Another question is only what is the K-nearest neighbor, I understand that when K = 1, then this is just a basic classification (classifies the instance to the class of the nearest neighbor). Can someone give me an idea on what classification the task of a 3-closest neighbor will definitely exceed the 1-closest neightbour classifier?

Thanks in advance!

Answer 1

k-NN vs decision tree

I always think that an image is the best way to get an intuition of an algorithm. The objective function that you offer will lead to the creation of a dataset like this:

x1 - x2 = 0. , , . , , , :

, , , , , , .

, , , , StackOverflow ( ).

, , , ,

k k-NN

, k k- , , k . , k , .

k-NN, . k = 1 k-NN :

k, , . , - :

k . , , , , ?

, , , , , , , ( ). , , , , , . , , (, , ..)

Answer 2

.

( , , , .)

, - , kNN, , k = 3 ( k = 1), , ( , ).

. , kNN, , , , , . - . . , " ", , . , (, ), , . , k = 3 , k = 1, , , , .

. , k = 3 , k = 1 , . , , , , , , , , .

, kNN - , , , ..

, .