標籤：

Pytorch手撕經典網路之LeNet5

05-15

下圖為經典網路LeNet5的網路結構。

具體講解可見卷積神經網路(CNN)學習筆記1：基礎入門。

實現時，主要包括倆個卷積層，倆個pooling層，三個全連接層（嚴格來說，最後一層的Gaussian connection應有其它的轉化方式，這裡用全連接）。

這裡面採用了mnist數據集，但為了更深入的學習pytorch，所以這裡採用了自定義數據源的方式。

下面一段代碼為載入mnist數據集，主要是解析二進位文件轉成numpy格式數據的過程。數據集是從mnist官方下載後的壓縮包，放置在工程代碼的data目錄下。

import gzip, structimport numpy as npdef _read(image,label): minist_dir = ./data/ with gzip.open(minist_dir+label) as flbl: magic, num = struct.unpack(">II", flbl.read(8)) label = np.fromstring(flbl.read(), dtype=np.int8) with gzip.open(minist_dir+image, rb) as fimg: magic, num, rows, cols = struct.unpack(">IIII", fimg.read(16)) image = np.fromstring(fimg.read(), dtype=np.uint8).reshape(len(label), rows, cols) return image,labeldef get_data(): train_img,train_label = _read( train-images-idx3-ubyte.gz, train-labels-idx1-ubyte.gz) test_img,test_label = _read( t10k-images-idx3-ubyte.gz, t10k-labels-idx1-ubyte.gz) return [train_img,train_label,test_img,test_label]

為了方便看某些圖片，這裡簡單實現了圖片列印的功能：

import matplotlib.pyplot as plt%matplotlib inlineX, y, Xt, yt = get_data()def imshow(img, label): plt.imshow(img.reshape((28,28))) plt.title(label)imshow(X[0], y[0])

接下來是用pytorch實現LeNet的部分。這部分較為簡單，對pytorch有了解後，按照LeNet的結構，按照步驟實現即可，需要注意的是由於LeNet處理的默認輸入時32*32的圖片，這裡加padding=2，即上下左右各padding 2個單位像素，擴充到32*32。

from torch import nfrom torch.nn import functional as Ffrom torch.autograd import Variableimport torchclass LeNet5(nn.Module): def __init__(self): super().__init__() self.conv1 = nn.Conv2d(1, 6, 5, padding=2) self.conv2 = nn.Conv2d(6, 16, 5) self.fc1 = nn.Linear(16*5*5, 120) self.fc2 = nn.Linear(120, 84) self.fc3 = nn.Linear(84, 10) def forward(self, x): x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) x = F.max_pool2d(F.relu(self.conv2(x)), (2, 2)) x = x.view(-1, self.num_flat_features(x)) x = F.relu(self.fc1(x)) x = F.relu(self.fc2(x)) x = self.fc3(x) return x def num_flat_features(self, x): size = x.size()[1:] num_features = 1 for s in size: num_features *= s return num_features

然後是訓練加預測的過程，本人喜歡邊訓練邊測試的過程，所以按照這樣的結構參照官方一些例子進行了實現。

這裡custom_normalization是手工實現標準化的過程（貌似不用，效果也沒差太多）。最後的準確率在99%以上。

#使用pytorch封裝的dataloader進行訓練和預測from torch.utils.data import TensorDataset, DataLoaderfrom torchvision import transformsdef custom_normalization(data, std, mean): return (data - mean) / stduse_gpu = torch.cuda.is_available()batch_size = 256kwargs = {num_workers: 2, pin_memory: True} if use_gpu else {}X, y, Xt, yt = get_data()#主要進行標準化處理mean, std = X.mean(), X.std()X = custom_normlization(X, mean, std)Xt = custom_normlization(Xt, mean, std)train_x, train_y = torch.from_numpy(X.reshape(-1, 1, 28, 28)).float(), torch.from_numpy(y.astype(int))test_x, test_y = [ torch.from_numpy(Xt.reshape(-1, 1, 28, 28)).float(), torch.from_numpy(yt.astype(int)) ]train_dataset = TensorDataset(data_tensor=train_x, target_tensor=train_y)test_dataset = TensorDataset(data_tensor=test_x, target_tensor=test_y)train_loader = DataLoader(dataset=train_dataset, shuffle=True, batch_size=batch_size, **kwargs)test_loader = DataLoader(dataset=test_dataset, shuffle=True, batch_size=batch_size, **kwargs)model = LeNet5()if use_gpu: model = model.cuda() print(USE GPU)else: print(USE CPU)criterion = nn.CrossEntropyLoss(size_average=False)optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, betas=(0.9, 0.99))def weight_init(m):# 使用isinstance來判斷m屬於什麼類型 if isinstance(m, nn.Conv2d): import math n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels m.weight.data.normal_(0, math.sqrt(2. / n)) elif isinstance(m, nn.BatchNorm2d):# m中的weight，bias其實都是Variable，為了能學習參數以及後向傳播 m.weight.data.fill_(1) m.bias.data.zero_()model.apply(weight_init)def train(epoch): model.train() for batch_idx, (data, target) in enumerate(train_loader): if use_gpu: data, target = data.cuda(), target.cuda() data, target = Variable(data), Variable(target) optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() optimizer.step() if batch_idx % 100 == 0: print(Train Epoch: {} [{}/{} ({:.0f}%)] Loss: {:.6f}.format( epoch, batch_idx * len(data), len(train_loader.dataset), 100. * batch_idx / len(train_loader), loss.data[0]))def test(): model.eval() test_loss = 0 correct = 0 for data, target in test_loader: if use_gpu: data, target = data.cuda(), target.cuda() data, target = Variable(data, volatile=True), Variable(target) output = model(data) test_loss += criterion(output, target).data[0] # sum up batch loss pred = output.data.max(1, keepdim=True)[1] # get the index of the max log-probability correct += pred.eq(target.data.view_as(pred)).cpu().sum() test_loss /= len(test_loader.dataset) print(
Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.2f}%)
.format( test_loss, correct, len(test_loader.dataset), 100. * correct / len(test_loader.dataset)))for epoch in range(1, 501): train(epoch) test()

完整代碼見sloth2012/LeNet5

本文實現lenet與原paper還是有些不一樣，主要體現在評論區說的s2到c3的過程上。對於該問題，本文實現算是一個簡化版本。原paper之所以那樣實現，也是受限於當時的計算資源。現簡化版本也符合pytorch的實現框架。pytorch原生並沒有提供s2到c3的計算過程模塊，用戶可自行實現，詳細可借鑒tiny-dnn/tiny-dnn。

參考鏈接

  • python讀取MNIST image數據
  • jeyzhang.com/cnn-learni
  • LeNet with Pytorch | Kaggle

推薦閱讀：

TAG:PyTorch | 深度學習DeepLearning |