Which one is faster? Logistic regression or linear-core SVM?

Question

I do machine learning with python (scikit-learn) using the same data, but with different classifiers. When I use 500 thousand. Data, LR and SVM (linear core) take about the same time, SVM (with polynomial core) takes forever. But using 5 million data, it seems that LR is faster than SVM (linear), it is very interesting, is this really what people usually find?

Answer 1

Faster is a bit of a weird question, partly because it’s hard to compare apples with apples, and it depends on the context. LR and SVM are very similar in the linear case. The TL; DR for the linear case is that logistic regression and SVM are very fast, and the speed difference is usually not too large, and in some cases they can be faster / slower.

From a mathematical point of view, logistic regression is strictly convex (its loss is also smoother), where SVMs are only convex, therefore LR helps to be “faster” in terms of optimization, but this does not always mean a faster expression of how long you wait.

, , , SVM . exp, , , max, SVM, . SVM , "", ( ), / . SVM C , LR .

, , - , . , , .

Dual Coordinate Descent, , , SVM, ( 2x). , LR SVM, , "", . , SMO SVM , , , , .