Reports on Progress:

Accuracy of Qulity Adviser

The accuracy of the Quality Advisor can be estimated by plotting the actual scores of audio quality (obtained from real listeners) against the predicted scores - see Fig. 1. There is a large correlation between the actual and predicted scores (R = 0.926) and the small standard deviation of residuals with respect to the scale (e = 9 %). However, there are still few outliers indicating that the model is not ideal and need improvement (maximum error = 33 %).

Fig. 1 Scatter plot of the actual scores plotted against the predicted scores

Fig. 2 shows a histogram of the residuals (differences between the actual and the predicted scores). Since it has a normal distribution with the standard deviation equal to 9 it is possible to estimate, according to the Chebychev's theorem, that 95 % of all absolute errors are less than 18 on a 100-point scale. It indicates that the model used in the Quality Advisor is not bad but it needs a further improvement.

Fig.2 Histogram of residuals (differences between actual and predicted scores)

For more information about the validation of the Quality Adviser click here.