Cross validation, recall and f1 with sklearn

Question

Is there an easy way to cross-check the classifier and calculate accuracy and recall right away? I am currently using the function

cross_validation.cross_val_score(classifier, designMatrix, classes, cv=5, scoring="precision") 

however, it only calculates one metric, so I have to call it 2 times in order to calculate the accuracy and call. With the large ML model, the calculation then excessively takes 2 times longer. Is there a built-in better option, or do I need to cross-check myself? thanks.

Answer 1

I'm not sure about the current state of affairs (this feature was discussed), but you can always get away with the following - terrible - hack

 from sklearn.metrics import recall_score, precision_score from sklearn.metrics.scorer import make_scorer recall_accumulator = [] def score_func(y_true, y_pred, **kwargs): recall_accumulator.append(recall_score(y_true, y_pred, **kwargs)) return precision_score(y_true, y_pred, **kwargs) scorer = make_scorer(score_func) 

Then use scoring=scorer in your cross validation. You should find the recall values ​​in the recall_accumulator array. However, note that this array is global, so make sure you don't write it the way you cannot interpret the results.

Answer 2

Eickenberg's answer works when the n_job argument of cross_val_score() set to 1. To support parallel computing (n_jobs> 1), you need to use a common list instead of a global list. This can be done using the Manager class from the multiprocessor module.

 from sklearn.metrics import precision_recall_fscore_support from sklearn.metrics.scorer import make_scorer from multiprocessing import Manager recall_accumulator = Manager().list() def score_func(y_true, y_pred, **kwargs): recall_accumulator.append(precision_recall_fscore_support(y_true, y_pred)) return 0 scorer = make_scorer(score_func) 

Then the result of each fold will be saved in recall_accumulator .