I have a few invigoration potions available but it doesn't let me use them during traces.
Welcome to the Community @OpheliaPeach! Invigoration Draught grants you Focus for casting Strategic Spells in the Wizarding Challenges, and you can only use these Potions during a Wizarding Challenge.
@OpheliaPeach Clarification in case you didn't understand @HPWUblue - A wizarding challenge is a fortress battle. You can only use them inside fortresses (the big towers).