Complexity Penalties in Statistical Learning

2019-02-06T04:13:57.140Z · score: 34 (12 votes)
Comment by michael_h on Complexity Penalties in Statistical Learning · 2019-02-06T04:07:22.038Z · score: 3 (2 votes) · LW · GW

Good point. Thank you for bringing this up. I just had a closer look in my notes at how the complexity penalty is derived and there is a additional assumption that I left out.

The derivation uses a matrix with columns and rows which has entry in the row and column (where is the training set). In the derivation it is assumed that has rank which true most of the time provided that . For simplicity I won't add a mention of this matrix to original post but I will add the assumption .