1.线性回归

Liner Regression

trainSet: $(x^{(i)},y^{(i)})$

input: $x$

output: $y$

model: $f_{w,b}(x)=wx+b$

cost funcation: $J(w,b)=\frac{1}{2m}\sum _{i=1}^m(f_{w,b}(x^{(i)})-y^{(i)}{)}^2$

gradient descent: ${\min }_{w,b}J(w,b)$

$$\begin{array}{rl}{w}^{\prime }& :=w-\alpha \frac{\partial }{\partial w}J(w,b)\\ & :=w-\alpha \frac{1}{2m}\sum _{i=1}^m\frac{\partial }{\partial w}(f_{w,b}(x^{(i)})-y^{(i)}{)}^2\\ & :=w-\alpha \frac{1}{m}\sum _{i=1}^m[(f_{w,b}(x^{(i)})-y^{(i)})x^{(i)}]\end{array}$$ $$\begin{array}{rl}{b}^{\prime }& :=b-\alpha \frac{\partial }{\partial b}J(w,b)\\ & :=b-\alpha \frac{1}{2m}\sum _{i=1}^m\frac{\partial }{\partial b}(f_{w,b}(x^{(i)})-y^{(i)}{)}^2\\ & :=b-\alpha \frac{1}{m}\sum _{i=1}^m(f_{w,b}(x^{(i)})-y^{(i)})\end{array}$$

$w=w^{\prime },b=b^{\prime }$ (Simultaneous update)

learning rate: $\alpha$

Multiple features(variables)

using vectorization

trainSet: $({\mathbf{x}}^{(i)},y^{(i)})$

input: $\mathbf{x}$

output: $y$

model: $f_{w,b}(x)={\mathbf{w}}^T\mathbf{x}+b$

cost funcation: $J(\mathbf{w},b)=\frac{1}{2m}\sum _{i=1}^m(f_{\mathbf{w},b}({\mathbf{x}}^{(i)})-y^{(i)}{)}^2$

gradient descent: ${\min }_{\mathbf{w},b}J(\mathbf{w},b)$

$$\begin{array}{rl}& \\ & \mathrm{for}\mathrm{j}=0\dots \mathrm{n}-1\\ w_j^{\prime }& :=w_j-\alpha \frac{\partial }{\partial w_j}J(\mathbf{w},b)\\ & :=w_j-\alpha \frac{1}{2m}\sum _{i=1}^m\frac{\partial }{\partial w_j}(f_{\mathbf{w},b}({\mathbf{x}}^{(i)})-y^{(i)}{)}^2\\ & :=w_j-\alpha \frac{1}{m}\sum _{i=1}^m[(f_{\mathbf{w},b}({\mathbf{x}}^{(i)})-y^{(i)})x_j^{(i)}]\end{array}$$ $$\begin{array}{rl}{b}^{\prime }& :=b-\alpha \frac{\partial }{\partial b}J(\mathbf{w},b)\\ & :=b-\alpha \frac{1}{2m}\sum _{i=1}^m\frac{\partial }{\partial b}(f_{\mathbf{w},b}({\mathbf{x}}^{(i)})-y^{(i)}{)}^2\\ & :=b-\alpha \frac{1}{m}\sum _{i=1}^m(f_{\mathbf{w},b}({\mathbf{x}}^{(i)})-y^{(i)})\end{array}$$

$\mathbf{w}={\mathbf{w}}^{\prime },b=b^{\prime }$ (Simultaneous update)

learning rate: $\alpha$