BatchNorm tracked biased running variance
Training correctly normalized with the biased batch variance, but that same estimate was also stored in running_var, making evaluation diverge from PyTorch. Running stats now use the unbiased estimate while training behavior stays unchanged, with coverage across 2D, 3D, and 4D inputs.
@@ BatchNorm.__call__ @@
mean, var = self._calc_stats(x)
if self.training and self.track_running_stats:
mu = self.momentum
+ _, running_var = self._calc_stats(x, ddof=1)
self.running_mean = (1 - mu) * self.running_mean + mu * mean
- self.running_var = (1 - mu) * self.running_var + mu * var
+ self.running_var = (1 - mu) * self.running_var + mu * running_var