Like think of this.. You have to wire a house for a job because you're an electrician. In real life, you wake up, make a plan, eat breakfast, gather your tools, make sure you have supplies for the entire day (because if you have to drive to go get stuff during the day its wasted time), work, break for lunch maybe, work, finish the day, eat and come home in order to prepare for the next day.
Now i obviously know this cant work like that because I may have to adjust priorities in game, but how it works now is a villager wakes up, eats, checks priorities, sees 10 linen needs to be made, brings a handful of items to a work station, completes....1 linen, checks priority, ((( And here is the killer ))) If their priority changes, or they didn't bring enough supplies for more work, they interrupt all of their work to either change priority, in which they walk all the items they had in the first place BACK to storage before they start working on a different priority. OR they didn't bring enough supplies in the first place to their work station, so they walk back to storage, get a handful of items and cycle all over again.
It makes for all this walking around. And distance also has no bearing on priority i've noticed.