Quartz

动手学深度学习v2

8分钟阅读

前置

课程内容：

  • 深度学习基础：线性神经网络，多层感知机
  • 卷积神经网络：LeNet, AlexNet, VGG, Inception, ResNet
  • 循环神经网络：RNN,GRU, LSTM， seq2seq
  • 注意力机制：Attention，Transformer
  • 优化算法：SGD，Momentum，Adam，
  • 高性能计算：并行，多GPU，分布式
  • 计算机视觉：目标检测，语义分割
  • 自然语言处理：词嵌入，BERT

环境：

python3.8
miniconda
pip install jupyter d2l torch torchvision
wget https://zh-v2.d2l.ai/d2l-zh.zip
ssh -L8888:localhost:8888 [email protected]

数据操作

  • 0d 一个类别
  • 1d 一个特征向量
  • 2d 一个样本的特征矩阵
  • 3d 一个 RGB 图片（width，height，channel）
  • 4d 一批量 RGB 图片（batch，width，height，channel）
  • 5d 一批量视频（batch，time，width，height，channel）

tensor 几乎是沿用了 numpy.ndarray，包含基本的构造函数、运算、索引切片

连结运算：

torch.cat((X, Y), dim=0), torch.cat((X, Y), dim=1)

广播机制：行列不相同的两个 tensor 通过复制部分元素使得具有相同的 shape 进行计算

尽量使用原地计算的方式（+=，-=）节省内存（节省内存申请和释放的时间）

线性代数

维度：对于向量来说，维度指向量的长度；对于张量来说，维度指轴的个数

范数：

np.linalg.norm(u)

矩阵计算

438

412406

自动微分

计算梯度

import torch
 
x = torch.arange(4.0)
x.requires_grad_(True)
y = 2 * torch.dot(x, x)
y.backward() 
x.grad # tensor([ 0.,  4.,  8., 12.])
x.gard == 4 * x

理解下方几个例子

y = x.sum()
####
y = x * x
y = y.sum()

线性神经网络

手搓线性回归

数据集：

def synthetic_data(w, b, num_examples):  #@save
    """生成y=Xw+b+噪声"""
    X = torch.normal(0, 1, (num_examples, len(w)))
    y = torch.matmul(X, w) + b
    y += torch.normal(0, 0.01, y.shape)
    return X, y.reshape((-1, 1))
 
true_w = torch.tensor([2, -3.4])
true_b = 4.2
features, labels = synthetic_data(true_w, true_b, 1000)
 
def data_iter(batch_size, features, labels):
    num_examples = len(features)
    indices = list(range(num_examples))
    # 这些样本是随机读取的，没有特定的顺序
    random.shuffle(indices)
    for i in range(0, num_examples, batch_size):
        batch_indices = torch.tensor(
            indices[i: min(i + batch_size, num_examples)])
        yield features[batch_indices], labels[batch_indices]

初始模型参数

w = torch.normal(0, 0.01, size=(2,1), requires_grad=True)
b = torch.zeros(1, requires_grad=True)
 
def linreg(X, w, b):  #@save
    """线性回归模型"""
    return torch.matmul(X, w) + b
    
def squared_loss(y_hat, y):  #@save
    """均方损失"""
    return (y_hat - y.reshape(y_hat.shape)) ** 2 / 2
 
def sgd(params, lr, batch_size):  #@save
    """小批量随机梯度下降"""
    with torch.no_grad():
        for param in params:
            param -= lr * param.grad / batch_size
            param.grad.zero_()

训练

lr = 0.03
num_epochs = 3
net = linreg
loss = squared_loss
 
for epoch in range(num_epochs):
    for X, y in data_iter(batch_size, features, labels):
        l = loss(net(X, w, b), y)  # X和y的小批量损失
        # 因为l形状是(batch_size,1)，而不是一个标量。l中的所有元素被加到一起，
        # 并以此计算关于[w,b]的梯度
        l.sum().backward()
        sgd([w, b], lr, batch_size)  # 使用参数的梯度更新参数
    with torch.no_grad():
        train_l = loss(net(features, w, b), labels)
        print(f'epoch {epoch + 1}, loss {float(train_l.mean()):f}')

softmax 回归（分类）

对输出使用独热编码 one-hot

交叉熵衡量两个概率的区别：

Huber’s Robust Loss

数据集：

def load_data_fashion_mnist(batch_size, resize=None):  # @save
    """下载Fashion-MNIST数据集，然后将其加载到内存中"""
    trans = [transforms.ToTensor()]
    if resize:
        trans.insert(0, transforms.Resize(resize))
    trans = transforms.Compose(trans)
    mnist_train = torchvision.datasets.FashionMNIST(root="../data", train=True, transform=trans, download=True)
    mnist_test = torchvision.datasets.FashionMNIST(root="../data", train=False, transform=trans, download=True)
    return (
        data.DataLoader(mnist_train, batch_size, shuffle=True, num_workers=MAX_WORKERS),
        data.DataLoader(mnist_test, batch_size, shuffle=False, num_workers=MAX_WORKERS),
    )
def get_fashion_mnist_labels(labels):  #@save
    """返回Fashion-MNIST数据集的文本标签"""
    text_labels = ['t-shirt', 'trouser', 'pullover', 'dress', 'coat',
                   'sandal', 'shirt', 'sneaker', 'bag', 'ankle boot']
    return [text_labels[int(i)] for i in labels]
    
batch_size = 256
train_iter, test_iter = load_data_fashion_mnist(batch_size)

参数：

num_inputs = 784
num_outputs = 10
W = torch.normal(0, 0.01, size=(num_inputs, num_outputs), requires_grad=True)
b = torch.zeros(num_outputs, requires_grad=True)
 
def softmax(X):
    X_exp = torch.exp(X)
    partition = X_exp.sum(1, keepdim=True) # 对每一行求和，保持维度
    return X_exp / partition  # 这里应用了广播机制
    
def net(X):
    return softmax(torch.matmul(X.reshape((-1, W.shape[0])), W) + b)
 
def cross_entropy(y_hat, y):
    return - torch.log(y_hat[range(len(y_hat)), y])
 
 def accuracy(y_hat, y):  #@save
    """计算预测正确的数量"""
    if len(y_hat.shape) > 1 and y_hat.shape[1] > 1:
        y_hat = y_hat.argmax(axis=1)
    cmp = y_hat.type(y.dtype) == y
    return float(cmp.type(y.dtype).sum())
 
def evaluate_accuracy(net, data_iter):  #@save
    """计算在指定数据集上模型的精度"""
    if isinstance(net, torch.nn.Module):
        net.eval()  # 将模型设置为评估模式
    metric = Accumulator(2)  # 正确预测数、预测总数
    with torch.no_grad():
        for X, y in data_iter:
            metric.add(accuracy(net(X), y), y.numel())
    return metric[0] / metric[1]

训练：

def train_epoch_ch3(net, train_iter, loss, updater):  #@save
    """训练模型一个迭代周期（定义见第3章）"""
    # 将模型设置为训练模式
    if isinstance(net, torch.nn.Module):
        net.train()
    # 训练损失总和、训练准确度总和、样本数
    metric = Accumulator(3)
    for X, y in train_iter:
        # 计算梯度并更新参数
        y_hat = net(X)
        l = loss(y_hat, y)
        if isinstance(updater, torch.optim.Optimizer):
            # 使用PyTorch内置的优化器和损失函数
            updater.zero_grad()
            l.mean().backward()
            updater.step()
        else:
            # 使用定制的优化器和损失函数
            l.sum().backward()
            updater(X.shape[0])
        metric.add(float(l.sum()), accuracy(y_hat, y), y.numel())
    # 返回训练损失和训练精度
    return metric[0] / metric[2], metric[1] / metric[2]
 
def train_ch3(net, train_iter, test_iter, loss, num_epochs, updater):  #@save
    """训练模型（定义见第3章）"""
    for epoch in range(num_epochs):
        train_metrics = train_epoch_ch3(net, train_iter, loss, updater)
        test_acc = evaluate_accuracy(net, test_iter)
        print(epoch + 1, train_metrics + (test_acc,))
    train_loss, train_acc = train_metrics
    assert train_loss < 0.5, train_loss
    assert train_acc <= 1 and train_acc > 0.7, train_acc
    assert test_acc <= 1 and test_acc > 0.7, test_acc

感知机

class Perceptron:
    def __init__(self, learning_rate=0.1, max_epochs=1000):
        self.lr = learning_rate
        self.max_epochs = max_epochs
        self.w = None
        self.b = None
        self.history = []
 
    def fit(self, X, y):
        n_samples, n_features = X.shape
        self.w = np.zeros(n_features)
        self.b = 0.0
        self.history = []
 
        for epoch in range(self.max_epochs):
            misclassified = 0
            for i in range(n_samples):
                if y[i] * (np.dot(self.w, X[i]) + self.b) <= 0:
                    self.w += self.lr * y[i] * X[i]
                    self.b += self.lr * y[i]
                    misclassified += 1
            self.history.append({
                'w': self.w.copy(),
                'b': self.b,
                'misclassified': misclassified
            })
            if misclassified == 0:
                print(f'在第 {epoch + 1} 轮训练后收敛，所有样本分类正确')
                return
        print(f'达到最大轮次 {self.max_epochs}，仍有 {misclassified} 个误分类')
 
    def predict(self, X):
        return np.sign(np.dot(X, self.w) + self.b)
文件名（File）：下载（Download）：
read meReadme.txt
sub infoSub_info.txt
64-channels64-channels.loc
freqency phaseFreq_Phase.mat
S1.matS1.mat.7z
S2.matS2.mat.7z
S3.matS3.mat.7z
S4.matS4.mat.7z
S5.matS5.mat.7z
S6.matS6.mat.7z
S7.matS7.mat.7z
S8.matS8.mat.7z
S9.matS9.mat.7z
S10.matS10.mat.7z
S11.matS11.mat.7z
S12.matS12.mat.7z
S13.matS13.mat.7z
S14.matS14.mat.7z
S15.matS15.mat.7z
S16.matS16.mat.7z
S17.matS17.mat.7z
S18.matS18.mat.7z
S19.matS19.mat.7z
S20.matS20.mat.7z
S21.matS21.mat.7z
S22.matS22.mat.7z
S23.matS23.mat.7z
S24.matS24.mat.7z
S25.matS25.mat.7z
S26.matS26.mat.7z
S27.matS27.mat.7z
S28.matS28.mat.7z
S29.matS29.mat.7z
S30.matS30.mat.7z
S31.matS31.mat.7z
S32.matS32.mat.7z
S33.matS33.mat.7z
S34.matS34.mat.7z
S35.matS35.mat.7z
文件名（File）：下载（Download）：
notenote.pdf
descriptiondescription.pdf
S1~S10S1-S10.tar.gz
S11~S20S11-S20.tar.gz
S21~S30S21-S30.tar.gz
S31~S40S31-S40.tar.gz
S41~S50S41-S50.tar.gz
S51~S60S51-S60.tar.gz
S61~S70S61-S70.tar.gz
文件名（File）：下载（Download）：
notenote.pdf
descriptiondescription.pdf
S1~S10S1-S10.tar.gz
S11~S20S11-S20.tar.gz
S21~S30S21-S30.tar.gz
S31~S40S31-S40.tar.gz
S41~S50S41-S50.tar.gz
S51~S60S51-S60.tar.gz
S61~S70S61-S70.tar.gz
文件名（File）：下载（Download）：
DescriptionDescription.tar.gz
S1~S10S1-S10.tar.gz
S11~S20S11-S20.tar.gz
S21~S30S21-S30.tar.gz
S31~S40S31-S40.tar.gz
S41~S50S41-S50.tar.gz
S51~S60S51-S60.tar.gz
S61~S70S61-S70.tar.gz
S71~S80S71-S80.tar.gz
S81~S90S81-S90.tar.gz
S91~S100S91-S100.tar.gz
文件名（File）：下载（Download）：
DescriptionDescription.txt
S1~S10S1-S10.mat.zip
S11~S14S11-S14.mat.zip
文件名（File）：下载（Download）：
64-channels64-channels.loc
notenote.txt
ReadmeReadme.txt
subjects_informationsubjects_information.xlsx
S1~S10S1-S10.mat.zip
S11~S20S11-S20.mat.zip
S21~S30S21-S30.mat.zip
S31~S40S31-S40.mat.zip
S41~S50S41-S50.mat.zip
S51~S60S51-S60.mat.zip
S61~S64S61-S64.mat.zip
文件名（File）：下载（Download）：
62-channels62-channels.loc
DescriptionDescription.pdf
sub_infosub_info.txt
ImageImage.zip
G1D1D2G1D1D2.zip
G2D1D2G2D1D2.zip
G3D1D2G3D1D2.zip
G4D1D2G4D1D2.zip
G5D1D2G5D1D2.zip
G6D1D2G6D1D2.zip
G7D1D2G7D1D2.zip
文件名（File）：下载（Download）：
ReadmeReadme.pdf
Stimulation informationstimulation_information.pdf
Subjects informationsubjects_information.mat
ImpedanceImpedance.mat
S1~S10S001-S010.zip
S11~S20S011-S020.zip
S21~S30S021-S030.zip
S31~S40S031-S040.zip
S41~S50S041-S050.zip
S51~S60S051-S060.zip
S61~S70S061-S070.zip
S71~S80S071-S080.zip
S81~S90S081-S090.zip
S91~S102S091-S102.zip
文件名（File）：下载（Download）：
ReadmeReadme.pdf
Code Word TablereqCodeword.mat
Offline experiments S1-S4S1-S4.rar
Offline experiments S5-S8S5-S8.rar
Online experiments SS1-SS4SS1-SS4.rar
Online experiments SS5-SS8SS5-SS8.rar
Online experiments SS9-SS12SS9-SS12.rar
文件名（File）：下载（Download）：
Readmereadme.pdf
Experiment 1: S1-S4S1-S4.zip
Experiment 1: S5-S8S5-S8.zip
Experiment 1: S9-S12S9-S12.zip
Experiment 2: S1-S2S1-S2.zip
Experiment 2: S3-S4S3-S4.zip
Experiment 2: S5-S6S5-S6.zip
Experiment 2: S7-S8S7-S8.zip
Experiment 2: S9-S10S9-S10.zip
Experiment 2: S11-S12S11-S12.zip
Experiment 3: S1-S2S1-S2.zip
Experiment 3: S3-S4S3-S4.zip
Experiment 3: S5-S6S5-S6.zip
Experiment 3: S7-S8S7-S8.zip
Experiment 3: S9-S10S9-S10.zip
Experiment 3: S11-S12S11-S12.zip
Experiment 4: S1-S4S1-S4.zip
Experiment 4: S5-S8S5-S8.zip
Experiment 4: S9-S12S9-S12.zip
文件名（File）：下载（Download）：
ReadmeREADME.pdf
DatasetAutismDetectionDataset.zip
文件名（File）：下载（Download）：
ReadmeREADME.txt
1 targetData of 1 Target.7z
40 targets offlineData of 40 Targets Offline.7z

若未能找到相应所需数据，请移至 清华大学脑机接口研究组(www.thubci.com) 资料下载页面下载或联系我们。

If you can not find the necessary data, please visit THU BCI Lab (www.thubci.com) Download page or contact us.

拖拽到此处完成下载

图片将完成下载

AIX智能下载器

关系图谱