2.逻辑回归

sigmoid function:

σ (z) = \frac{1}{1 + e ^{z}}, σ^{'} (z) = σ (z) (1 - σ (z))

trainSet: $(x^{(i)}, y^{(i)})$

input: $x$

output: $y \in 0, 1$

model:

z = w^{T} x + b f_{w, b} (x) = σ (z) = \frac{1}{1 + e ^{- (w^{T} x + b)}}

loss:

L (f_{w, b} (x^{(i)}), y^{(i)}) = {- lo g (f_{w, b} (x^{(i)})) - lo g (1 - f_{w, b} (x^{(i)})) if y^{(i)} = 1 if y^{(i)} = 0 = - y^{(i)} lo g (f_{w, b} (x^{(i)})) - (1 - y^{(i)}) lo g (1 - f_{w, b} (x^{(i)}))

cost funcation: $J (w, b) = \frac{1}{m} i = 1 \sum m L (f_{w, b} (x^{(i)}), y^{(i)})$

gradient descent: $min_{w, b} J (w, b)$

\frac{\partial}{\partial w _{j}} J (w, b) = \frac{1}{m} i = 1 \sum m \frac{\partial}{\partial w _{j}} L (f_{w, b} (x^{(i)}), y^{(i)}) = \frac{1}{m} i = 1 \sum m \frac{\partial}{\partial w _{j}} (- y^{(i)} lo g (f_{w, b} (x^{(i)})) - (1 - y^{(i)}) lo g (1 - f_{w, b} (x^{(i)})) = \frac{1}{m} i = 1 \sum m [- y^{(i)} \frac{\frac{\partial}{\partial w _{j}} f _{w, b} ( x ^{(i)} )}{f _{w, b} ( x ^{(i)} )} - (1 - y^{(i)}) \frac{- \frac{\partial}{\partial w _{j}} f _{w, b} ( x ^{(i)} )}{1 - f _{w, b} ( x ^{(i)} )}] = \frac{1}{m} i = 1 \sum m [- \frac{y ^{(i)}}{f _{w, b} ( x ^{(i)} )} + \frac{1 - y ^{(i)}}{1 - f _{w, b} ( x ^{(i)} )}] \frac{\partial}{\partial w _{j}} f_{w, b} (x^{(i)}) = \frac{1}{m} i = 1 \sum m [- \frac{y ^{(i)}}{f _{w, b} ( x ^{(i)} )} + \frac{1 - y ^{(i)}}{1 - f _{w, b} ( x ^{(i)} )}] f_{w, b} (x^{(i)}) (1 - f_{w, b} (x^{(i)})) \frac{\partial}{\partial w _{j}} (w^{T} x + b) = \frac{1}{m} i = 1 \sum m [- y^{(i)} (1 - f_{w, b} (x^{(i)})) + (1 - y^{(i)}) f_{w, b} (x^{(i)})] x_{j}^{(i)} = \frac{1}{m} i = 1 \sum m [f_{w, b} (x^{(i)}) - y^{(i)}] x_{j}^{(i)}

w_{j}^{'} b^{'} for j = 0 \dots n - 1 := w_{j} - α \frac{\partial}{\partial w _{j}} J (w, b) := w_{j} - α \frac{1}{m} i = 1 \sum m (f_{w, b} (x^{(i)}) - y^{(i)}) x_{j}^{(i)} := b - α \frac{\partial}{\partial b} J (w, b) := b - α \frac{1}{m} i = 1 \sum m (f_{w, b} (x^{(i)}) - y^{(i)})

Quartz