there should be a couple options between what items you do and don't want to scale, obviously totems are their own separate thing but the mace, swords and food should be able to be scaled separately too since they're the most common ones people want to have small.
whether it be a sub section of the held item scale or each one gets their one scale like the totem, I think it's a good addition as many texture packs don't always scale things the way somebody might want.