问题描述
我正在尝试在模型上使用特征重建和样式重建损失。为此,我遵循了PyTorch网站上“神经样式转换”的示例代码。
https://pytorch.org/tutorials/advanced/neural_style_tutorial.html
尽管可以毫无问题地计算出特征损失,但样式损失始终为零。而且我找不到原因,因为在执行过程中一切看起来都很好。这些损失函数的计算方法与所提出的数学方法相同。此外,如您所知,样式和特征损失在计算方面几乎相同,除了Gram矩阵逐步减少样式损失且特征损失没有问题。
有人能帮助我解决这种情况吗?
class Feature_and_style_losses():
def __init__(self,):
self.vgg_model = models.vgg19(pretrained=True).features.cuda().eval()
self.content_layers = ['conv_16']
self.style_layers = ['conv_5']
def calculate_feature_and_style_losses(self,input_,target,feature_coefficient,style_coefficient):
i = 0
feature_losses = []
style_losses = []
for layer_ in self.vgg_model.children():
if isinstance(layer_,nn.Conv2d):
i += 1
name = "conv_{}".format(i)
if name in self.content_layers:
features_input = self.vgg_model(input_).detach()
features_target = self.vgg_model(target).detach()
feature_losses.append(self.feature_loss(features_input,features_target))
if name in self.style_layers:
style_input = self.vgg_model(input_).detach()
style_target = self.vgg_model(target).detach()
style_losses.append(self.style_loss(style_input,style_target))
feature_loss_value = (torch.mean(torch.from_numpy(np.array(feature_losses,dtype=np.float32)))) * feature_coefficient
style_loss_value = (torch.mean(torch.from_numpy(np.array(style_losses,dtype=np.float32)))) * style_coefficient
return feature_loss_value,style_loss_value
def feature_loss(self,target):
target = target.detach()
feature_reconstruction_loss = F.mse_loss(input_,target)
return feature_reconstruction_loss
def gram_matrix(self,input_):
a,b,c,d = input_.size() #??? check size
features = input_.view(a*b,c*d)
#features_t = features.transpose(1,2)
#G = features.bmm(features_t) / (b*c*d)
#print(features.shape)
G = torch.mm(features,features.t())
return G.div(a*b*c*d)
return G
def style_loss(self,target):
G_input = self.gram_matrix(input_)
G_target = self.gram_matrix(target).detach()
#style_reconstruction_loss = self.feature_loss(G_input,G_target)
style_reconstruction_loss = F.mse_loss(G_input,G_target)
return style_reconstruction_loss
feature_loss_ = Feature_and_style_losses()
...
for e in range(epochs):
for i,batch in enumerate(DataLoader):
...
real_C = Variable(batch["C"].type(Tensor))
fake_C = independent_decoder(features_all)
f_loss,s_loss = feature_loss_.calculate_feature_and_style_losses(fake_C,real_C,1,10)
loss_G_3 = loss_GAN_3 + lambda_pixel * (loss_pixel_3_object + loss_pixel_3_scene) * 0.5 + f_loss + s_loss
loss_G_3.backward(retain_graph=True)
optimizer_independent_decoder.step()
最好。
解决方法
暂无找到可以解决该程序问题的有效方法,小编努力寻找整理中!
如果你已经找到好的解决方法,欢迎将解决方案带上本链接一起发送给小编。
小编邮箱:dio#foxmail.com (将#修改为@)