Appendix02 Proof of Multiple Linear Regression

$\hat{\beta}=$

$\begin{array}{ll} \because & \hat{\mathbf{y}} = \mathbf{X}\hat{\boldsymbol{\beta}}\\ \therefore & {\rm min}\big(\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}\big) = {\rm min}\big(\sum_{i=1}^{n}(y_{i}-\mathbf{x}_{i}\hat{\boldsymbol{\beta}})^{2}\big) \\ & = {\rm min}\big((\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}})^{T}\cdot(\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}})\big)\\ & = {\rm min}\big((\mathbf{y}^{T}-\hat{\boldsymbol{\beta}}^{T}\mathbf{X}^{T})\cdot(\mathbf{y}-\mathbf{X}\hat{\boldsymbol{\beta}})\big) \\ & = \text{min}\big(\mathbf{y}^{T}\mathbf{y} + \hat{\boldsymbol{\beta}}^{T}\mathbf{X}^{T}\mathbf{X}\hat{\boldsymbol{\beta}} - 2 \hat{\boldsymbol{\beta}}^{T}\mathbf{X}^{T}\mathbf{y}\big)\\ \therefore & \text{assume }f = \mathbf{y}^{T}\mathbf{y} + \hat{\boldsymbol{\beta}}^{T}\mathbf{X}^{T}\mathbf{X}\hat{\boldsymbol{\beta}} - 2 \hat{\boldsymbol{\beta}}^{T}\mathbf{X}^{T}\mathbf{y} \\ \therefore & \frac{\partial{f}}{\partial{\hat{\boldsymbol{\beta}}}} = \mathbf{0} \\ \because & \boldsymbol{\beta} \text{ is a vector} \\ \therefore & \text{use the Denominator layout} \\ \therefore & \frac{\partial{f}}{\partial{\hat{\boldsymbol{\beta}}}} = 0 + ((\mathbf{\mathbf{X}^{T}\mathbf{X}})^{\mathbf{T}}+\mathbf{\mathbf{\mathbf{X}^{T}\mathbf{X}}})\cdot\boldsymbol{\beta} - 2 \mathbf{X^{T}y} = \mathbf{0} \\ \therefore & \hat{\boldsymbol{\beta}} = (\mathbf{X^{T}X})^{-1}\mathbf{X^{T}y} \\ \end{array}$