Statistical Significant Testing in IR
Ben Carterette from University of Delaware had an interesting tutorial on ICTIR 2013 about Statistical Significant Testing in Theory and in Practice (the same one also talked on SIGIR 2014 as well). This should be read by any IR or ML researcher. One thing of particular interesting is multiple testing problem.