How can I not lose my host credentials after joining a breakout room?

Whenever I create breakout rooms for a small-mid size group (~25 participants) and then try to join one of those breakout rooms, I lose my host controls and get put back in the main room. Is this a known issue or something to do with how I am assigning participants to rooms? I tested with three total participants (myself included as host) and was able to join, leave, and join another room.