Revising our Evaluation Practices in Machine Learning

Associate Professor Nathalie Japkowicz

School of Information Technology and Engineering, University of Ottawa, Canada

Date and time: 12.30 - 13.30, Friday 3rd November, 2006

Venue: 12.05.02

Abstract:

The evaluation of classifiers or learning algorithms is not a topic that has, generally, been given much thought in the fields of Machine Learning and Data Mining. More often than not, common off-the-shelf metrics such as Accuracy, Precision/Recall and ROC Analysis as well as confidence estimation methods, such as the t-test, are applied without much attention being paid to their meaning. The purpose of this talk is to underline some of the problems that can arise from our current practices and suggest that it might be useful, in certain cases, to either borrow evaluation methods from other fields or create our own measures. We will specifically look at two different types of domains, domains in which the two classes are as important, and domains suffering from a severe class imbalance and show how some of the methods stemming from the field of biostatistics can be of use to us in these cases. We will conclude the talk by presenting a novel evaluation method designed for the severe class imbalance case.

About the speaker:

Dr. Nathalie Japkowicz is an Associate professor of Computer Science in the School of Information Technology and Engineering at the University of Ottawa. She is a visiting professor at Monash University, Clayton during the 2006-2007 Canadian School Year. She obtained her Ph.D. from Rutgers University (New Jersey) in 1999. Her area of study is Machine Learning with special emphases on the class imbalance problem, one-class learning, machine learning applied to computer and nuclear security, text mining, and, more recently, performance evaluation for machine learning.

Seminar Organisation

Seminars are free and open to the general public. No booking is necessary. If you are interested in giving a presentation in this seminar series, or to make suggestions for speakers, please contact Xiaodong Li, the seminar co-ordinator.