How to display scored probabilities from Machine Learning Toolkit model?

[cjcoleman]

I'm working with the Machine Learning Toolkit to build and test some binary classification models. I see that once the test set is scored, the predicted classification for the models are displayed but I am unable to locate the specifically predicted probabilities for these classifications.

Is it possible to display this result in addition to the predicted class? If so, how?

[yangzd · Accepted solution]

Predicted probabilities are supported in LogisticRegression. You can simply include probabilities=true in your fit query and it will show the predicted probabilities alongside the predicted classes.

Please refer to the MLTK documentation here: http://docs.splunk.com/Documentation/MLApp/latest/User/Algorithms#LogisticRegression

[cjcoleman]

Thank you, this is exactly what I was looking for.

[iceco]

Why that field doesn't show to me even I set probabilities=true? Can I know the field name of probabilities ?

[yangzd]

The generated fields will be something like predicted(target_field) for the class prediction, and bunch of probability(target_field=class_a), probability(target_field=class_b)... for the probabilities of predicting the fields as class_a, class_b, etc.
Can you double check if you are using LogisticRegression properly and at least seeing the predicted(target_field)?

[iceco]

Yes, I just figured it out. And when you want to apply it, you'd better apply "model_name
"probabilities=true to show it in the table

[yangzd]

That's great!

[iceco]

Thanks for your suggestion. And do you know do other algorithms have the similar function. I mean the toolkit uses scikit-learn lib, almost every classification model in this lib should have probability function, but I didn't see it in the MLTK doc

[aeapen]

@iceco It's quite easy to add probability function to other classifiers, by extending the existing ones using Custom functions. You can look at the code for LogisticRegression as an example, and add similar for others classifiers.

[yangzd]

Great point. Definitely other algorithms should support the probability prediction, as long as it's available in scikit-learn. Please stay tuned for the next MLTK release(s)!