Each minigame can be its own prefab that handles input independent of the "outside" player controller.
The root object of each minigame prefab can have some monobehiour that implements an interface or inherits from a super class for EnterGame() and ExitGame().
When the player interacts with a minigame, check for an available minigame (something like a raycast hitting an arcade box or whatever the minigame prefab will be). If there's an available minigame, cache it in an activeMinigame var, and set the outside player controller to stop taking movement inputs and whatever else you dont want to happen while the player is in a minigame. And call the EnterGame() method on the minigame to start it.
When the player presses the exit minigame button, call the ExitGame() method on the activeMinigame var you saved to pause/end it. And then set the activeMinigame var back to null and re-enable input.