I know they are not the same thing, but i am having trouble understanding why use .detach()
when .Eval() also stops the neural network from back propagation so the weights will not change
#.Eval() vs .Detach()
mellow drift
rugged spruce
Yes, if you're using the model for inference then going into eval() mode makes sure it doesn't recalculate the autograd and, very importantly, stops dropout etc. Note that detach() returns a new tensor with current values but not attached to the autograd so it never needs a gradient. Not that common to use detach() in SL I don't think.
unkempt token
Yes, if you're using the model for inference then going into eval() mode makes s...
detach/eval is used at times for GANs and/or .detach is used for freezing weights
eval may be better at times simply cuz its faster and such. so depending on your task it may be better to use eval
for example, having a frozen backbone:
with torch.inference_mode():
features = backbone.forward(x)
classify_out = head.forward(features)
will likely be faster than having detach