Masterarbeit Note-1

Transformer

• Transformer 的 PyTorch 实现
• Transformer 详解

Question

1. freq bias 需要梯度吗

with torch.no_grad():
     self.freq_bias = FrequencyBias(cfg.TRAIN.DATASETS[0])