I have observed very strange behavior when configuring SVM parameters with caret. When preparing a single model without tuning SVM with a radial core takes longer than SVM with a linear core, as expected. However, when configuring SVMs with both cores on the same penalty grid, SVMs with a linear kernel take significantly longer than SVMs with a radial core. This behavior can be easily reproduced on both Windows and Linux with R 3.2 and caret6.0-47. Does anyone know why setting up a linear SVM takes much longer than the radial core SVM core?
SVM linear
user system elapsed
0.51 0.00 0.52
SVM radial
user system elapsed
0.85 0.00 0.84
SVM linear tuning
user system elapsed
129.98 0.02 130.08
SVM radial tuning
user system elapsed
2.44 0.05 2.48
Toy example code below:
library(data.table)
library(kernlab)
library(caret)
n <- 1000
p <- 10
dat <- data.table(y = as.factor(sample(c('p', 'n'), n, replace = T)))
dat[, (paste0('x', 1:p)) := lapply(1:p, function(x) rnorm(n, 0, 1))]
dat <- as.data.frame(dat)
sigmas <- sigest(as.matrix(dat[, -1]), na.action = na.omit, scaled = TRUE)
sigma <- mean(as.vector(sigmas[-2]))
cat('\nSVM linear\n')
print(system.time(fit1 <- train(y ~ ., data = dat, method = 'svmLinear', tuneLength = 1,
trControl = trainControl(method = 'cv', number = 3))))
cat('\nSVM radial\n')
print(system.time(fit2 <- train(y ~ ., data = dat, method = 'svmRadial', tuneLength = 1,
trControl = trainControl(method = 'cv', number = 3))))
cat('\nSVM linear tuning\n')
print(system.time(fit3 <- train(y ~ ., data = dat, method = 'svmLinear',
tuneGrid = expand.grid(C = 2 ^ seq(-5, 15, 5)),
trControl = trainControl(method = 'cv', number = 3))))
cat('\nSVM radial tuning\n')
print(system.time(fit4 <- train(y ~ ., data = dat, method = 'svmRadial',
tuneGrid = expand.grid(C = 2 ^ seq(-5, 15, 5), sigma = sigma),
trControl = trainControl(method = 'cv', number = 3))))