A Machine Learning Life Cycle, (abbreviated ML Life Cycle) is a conceptual framework of the cyclical process that ML projects go through. However, there is no agreement on what its various phases are. Nevertheless, the following elements are popular. In consecutive order: Understand the problem Collect the data Wrangle the…
Performance Metrics Performance metrics tell you something about the performance of a machine learning model. Each metric has a specific focus. Because of the confusion matrix’ nature, a lot of metrics have a close sibling. Equally confusing is that many performance metrics have multiple synonyms, depending on the context. Given…
What is the Accuracy? The Accuracy is a performance metric that tells you the fraction of the predictions that were correct, without distinguishing between positive and negative predictions. The Accuracy can be a very misleading metric when the data set is unbalanced (when the prevalence is either very high or very…
What is Balanced Accuracy? Balanced Accuracy is a performance metric to evaluate a binary classifier. Why not use regular accuracy? Balanced accuracy is a better instrument for assessing models that are trained on data with very imbalanced target variables. I.e. very high, or very low prevalence. This will result in…
What is the Classification Success Index? The Classification Success Index (CSI) is a (fairly uncommon) measure for evaluating classifiers. The CSI focuses exclusively on the positive class. It is calculated as follows: The terms (1-PPV) and (1-TPR) correspond to the proportions of type I and type II errors. The measure…
What is the Diagnostic Odds Ratio? The Diagnostic Odds Ratio (DOR) is a performance metric to assess the effectiveness of a diagnostic test or — within the context of machine learning — a binary classification model. It is defined as the ratio of the odds of the prediction being positive is…
What is the F1-Score? The F1-Score has many names: F-Score F-Measure Sørensen’s Similarity Coefficient Sørensen–Dice Coefficient Dice Similarity Coefficient (DSC) Dice’s Coincidence Index Hellden’s Mean Accuracy Index The F1-Score is a metric to evaluate the performance of a binary classifier. It is calculated as the harmonic mean of the precision…
What is the False Discovery Rate? Given a positive prediction, the False Discovery Rate (FDR) is the performance metric that tells you the probability that the true value is negative. It is closely related to the False Omission Rate, which is completely analogous. The complement of the False Discovery Rate…
What is the False Negative Rate? The False Negative Rate (Miss Rate) is a performance metric that measures the probability that your model will predict negative when the true value is positive. It is closely related to the False Positive Rate, which is completely analogous. The True Positive Rate and the…
What is the False Omission Rate? Given a negative prediction, the False Omission Rate (FDR) is the performance metric that tells you the probability that the true value is positive. It is closely related to the False Discovery Rate, which is completely analogous. The complement of the False Omission Rate…