How to pass argument to scoring function in scikit-learn LogisticRegressionCV call

Question

Problem

I am trying to use scikit-learn LogisticRegressionCVwith roc_auc_scoreas a score indicator.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

clf = LogisticRegressionCV(scoring=roc_auc_score)

But when I try to fit the model ( clf.fit(X, y)), it throws an error.

 ValueError: average has to be one of (None, 'micro', 'macro', 'weighted', 'samples')

That's cool. It is clear what is happening: roc_auc_scoreit is necessary to call with the argument averagespecified in its documentation and the above error. So I tried this.

clf = LogisticRegressionCV(scoring=roc_auc_score(average='weighted'))

But it turns out that roc_auc_scoreyou cannot call only with an optional argument, because it causes another error.

TypeError: roc_auc_score() takes at least 2 arguments (1 given)

Question

Any thoughts on how I can use it roc_auc_scoreas a measure of the score LogisticRegressionCVso that I can specify an argument for the scoring function?

SO- scikit-learn GitHub repo, - ?

Answer 1

!

scikit-learn make_scorer metrics, , , ( . scikit-learn).

, average.

roc_auc_weighted = sk.metrics.make_scorer(sk.metrics.roc_auc_score, average='weighted')

LogisticRegressionCV, - !

clf = LogisticRegressionCV(scoring=roc_auc_weighted)

Answer 2

make_scorer, .

from sklearn.linear_model import LogisticRegressionCV
from sklearn.metrics import roc_auc_score, make_scorer
from sklearn.datasets import make_classification

# some example data
X, y = make_classification()

# little hack to filter out Proba(y==1)
def roc_auc_score_proba(y_true, proba):
    return roc_auc_score(y_true, proba[:, 1])

# define your scorer
auc = make_scorer(roc_auc_score_proba, needs_proba=True)

# define your classifier
clf = LogisticRegressionCV(scoring=auc)

# train
clf.fit(X, y)

# have look at the scores
print clf.scores_