heady helm
Hey Team, we've worked on a Project to teach LLMs to learn Math. As we all know, even the SOTA Models cannot do Math operations such as Multiplication for >3 digits without using a calculator.
Our intuition behind why models cannot do Math is because the instructions on the internet are something like a x b = c and do not follow the procedure which we humans follow when doing Math. For example when asked any human how to do 123 x 45, we follow the digit wise multiplication technique using carry, get results for each digit multiplication and then add the corresponding resulting numbers.
We've used a similar technique to do Math as part of our project. We've actually created a Synthetic Data of 1300 instructions for the step-by-step multiplication, Finetuned the SOTA Models like ChatGpt3.5 Turbo and we're able to get 98.5% Accuracy when doing Math Tasks.
As part of existing literature, people have used complex approaches to do Math such as creating an LLM only for math, using 50M instructions etc, Using complex approaches such as RL etc. We avoid all of this and only do traditional multiplication and we're able to see good Accuracy and also the validation loss and training loss with our Approach go close to 0 within 0.1 epochs.
We need AI Community help in providing us Feedback, checking our Overall Approach etc so sharing it here
glossy sundial
soft arch
zenith wasp
ocean wedge
sage forum
quick haven