Short answer: it's because that's the way useState() works... It will initialize the state the FIRST time and then subsequent calls are essentially ignored.
In the same way that
const [count, setCount] = useState(0)
will not reset the count back to 0 with every re-render neither do your dice get reset...
Now, the reason you want the initializer function instead of calling the function directly is a little more subtle...
With this code block:
const [dice, setDice] = useState(generateAllNewDice());
this code is identical to the following:
const newDice = generateAllNewDice();
const [dice, setDice] = useState(newDice);
I hope you can see here that generateAllNewDice() will be called with EACH re-render but the values are not used because useState will simply ignore the values on re-renders (just as explained above).
However when you pass a function reference then that function is NOT immediately executed but rather only the reference is passed.
const [dice, setDice] = useState(() => generateAllNewDice());
In this case the reference to anonymous function is passed to useState BUT it's not called on re-renders.
This makes it very efficient because imagine if the generateAllNewDice() function where very expensive... then with the first way you'd be uselessly calling it with EVERY re-render only to have the result ignored. However with the second way you would only call the expensive function once.
As an aside, you can do the second way a little shorter like this:
const [dice, setDice] = useState(generateAllNewDice);
Note the lack of () - this passes the reference to getAllNewDice instead of executing the function and passing the results.