Wednesday, June 6, 2012

AUC Explanation

I got this email from David Lexer:
I'm currently learning Mahout (e.g. - 'Mahout in Action') and discovered your Blog.  Cool!
Question (if you don't mind too much):

I just finished the lectures from 'Learning from Data' @Caltech and am in week 8 of  'Machine Learning' @Stanford/Coursera.
Neither course discussed the 'AUC' metric for classification.
Do you use it?  What's the mathematical basis?
Thanks in Advance,

My answer:
I am not using AUC so often, but I find it useful for cases the prediction is binary (0/1) and you need to estimate your prediction accuracy.
Another example where AUC is useful is in this year ACM KDD CUP 2012, track 2 they used AUC to evaluate accuracy of predicting CTR (click through rate) of ads. They recommend the paperROC graphs: Notes and practical considerations for researchers by Tom Fawcett as a good resource for learning about AUC. 

Some more detailed explanation of AUC is found here,  along with a few more references.

By the way, if we brought up KDD CUP track2 this year, in a few days I will have some interesting news to report. As for now I can only hint by showing this image:

Stay tuned!

No comments:

Post a Comment