Hey guys👋
I'm new to ML and I'm writing my own AI library to understand how neural networks work (not without chatagpt, of course, but I'm learning everything!! And I can now write a simple but fully functional neural network from scratch without help).
But because of the AI's stupid answers, I still don't quite understand how mini-batching works.
I imagine it like this: 16 examples are forward-fed, a loss is calculated for each, but backpropagation and the optimizer are only applied when you average these losses.
I've built this batching into my library, but it only slightly impairs training.
The AI ​​is giving me complete nonsense about this. I also tried accumulating gradients before, but it didn't help at all; the entire training process just crashed.
I hope you understand, and thanks in advance for your reply!! I'll be very grateful.
#Loss function
coral beacon