使用PyTorch手动分配权重

Question

我正在使用Python 3.8和PyTorch 1.7手动分配和更改神经网络的权重和偏置。举个例子，我定义了一个LeNet-300-100全连接神经网络来对MNIST数据集进行训练。该类定义的代码如下：

class LeNet300(nn.Module):
    def __init__(self):
        super(LeNet300, self).__init__()
        
        # Define layers-
        self.fc1 = nn.Linear(in_features = input_size, out_features = 300)
        self.fc2 = nn.Linear(in_features = 300, out_features = 100)
        self.output = nn.Linear(in_features = 100, out_features = 10)
        
        self.weights_initialization()
    
    
    def forward(self, x):
        out = F.relu(self.fc1(x))
        out = F.relu(self.fc2(out))
        return self.output(out)
    
    
    def weights_initialization(self):
        '''
        When we define all the modules such as the layers in '__init__()'
        method above, these are all stored in 'self.modules()'.
        We go through each module one by one. This is the entire network,
        basically.
        '''
        for m in self.modules():
            if isinstance(m, nn.Linear):
                nn.init.xavier_normal_(m.weight)
                nn.init.constant_(m.bias, 0)

为了尝试更改此模型的权重，可以进行以下实验-

# Instantiate model-
mask_model = LeNet300()

为了将每层中的所有权重都指定为一（1），我使用以下代码 -

with torch.no_grad():
    for layer in mask_model.state_dict():
        mask_model.state_dict()[layer] = nn.parameter.Parameter(torch.ones_like(mask_model.state_dict()[layer]))

# Sanity check-
mask_model.state_dict()['fc1.weight']

这个输出表明权重不等于1。 我还尝试了以下代码-

for param in mask_model.parameters():
    # print(param.shape)
    param = nn.parameter.Parameter(torch.ones_like(param))

但这并不起作用。

需要帮助吗？

- Arun

Answer 1

for param in mask_model.parameters():
    param.data = nn.parameter.Parameter(torch.ones_like(param))

- user15440901

Answer 2

我用非常简单的方法实现了这个（只使用了 fill_()），以下是代码：

import torch
import torch.nn as nn
class LeNet300(nn.Module):
    def __init__(self):
        super(LeNet300, self).__init__()
        
        # Define layers-
        self.fc1 = nn.Linear(in_features = 28, out_features = 300)
        self.fc2 = nn.Linear(in_features = 300, out_features = 100)
        self.output = nn.Linear(in_features = 100, out_features = 10)
        
        self.weights_initialization()
    
    
    def forward(self, x):
        out = F.relu(self.fc1(x))
        out = F.relu(self.fc2(out))
        return self.output(out)
    
    
    def weights_initialization(self):
        '''
        When we define all the modules such as the layers in '__init__()'
        method above, these are all stored in 'self.modules()'.
        We go through each module one by one. This is the entire network,
        basically.
        '''
        for m in self.modules():
            if isinstance(m, nn.Linear):
                nn.init.xavier_normal_(m.weight)
                nn.init.constant_(m.bias, 0)


mask_model = LeNet300()


with torch.no_grad():
    for layer in mask_model.state_dict():
        print(layer)
        #print(torch.ones_like(mask_model.state_dict()[layer].data))
        mask_model.state_dict()[layer].data.fill_(1)


mask_model.state_dict()['fc1.weight']
#   tensor([[1., 1., 1.,  ..., 1., 1., 1.],
#        [1., 1., 1.,  ..., 1., 1., 1.],
#        [1., 1., 1.,  ..., 1., 1., 1.],
#        ...,
#        [1., 1., 1.,  ..., 1., 1., 1.],
#        [1., 1., 1.,  ..., 1., 1., 1.],
#        [1., 1., 1.,  ..., 1., 1., 1.]])

- Prajot Kuvalekar

Answer 3

如果我们需要将一个 numpy数组 赋值给 层权重，可以按照以下方法进行：

numpy_data= np.random.randn(6, 1, 3, 3)

conv = nn.Conv2d(1, 6, 3, 1, 1, bias=False)
with torch.no_grad():
    conv.weight = nn.Parameter(torch.from_numpy(numpy_data).float())
    # or
    conv.weight.copy_(torch.from_numpy(numpy_data).float())

来源：https://discuss.pytorch.org/t/how-do-i-pass-numpy-array-to-conv2d-weight-for-initialization/56595/3

如何将numpy数组传递给Conv2d权重进行初始化？

我有一个形状为（64, 3, 7, 7）的numpy数组，表示一个卷积层的权重。现在我想将其用作PyTorch中Conv2d层的权重。有没有办法可以实现这一点？

- Vijay Mariappan