I got this email from David Lexer:
I'm currently learning Mahout (e.g. - 'Mahout in Action') and discovered your Blog. Cool!
Question (if you don't mind too much):
I just finished the lectures from 'Learning from Data' @Caltech and am in week 8 of 'Machine Learning' @Stanford/Coursera.
Neither course discussed the 'AUC' metric for classification.
Do you use it? What's the mathematical basis?
Thanks in Advance,
I am not using AUC so often, but I find it useful for cases the prediction is binary (0/1) and you need to estimate your prediction accuracy.
Another example where AUC is useful is in this year ACM KDD CUP 2012, track 2 they used AUC to evaluate accuracy of predicting CTR (click through rate) of ads. They recommend the paper ‘ROC graphs: Notes and practical considerations for researchers‘ by Tom Fawcett as a good resource for learning about AUC.
Some more detailed explanation of AUC is found here, along with a few more references.
By the way, if we brought up KDD CUP track2 this year, in a few days I will have some interesting news to report. As for now I can only hint by showing this image: