No IVR during working hours, IVR for non-business hours.

You need to use a custom rule.
Phone System > Auto Receptionist > General Settings > Company Call Handling > Add Custom Rule

in creating the custom rule you have 3 steps.
1. name of the rule (up to you)
2. conditions for the rule.
- Here you need to choose the 2nd Option (Called number)
- and also choose the 3rd option ( choose your work hours)
3. call handling if conditions are met. Here you need to choose forward to front desk.

This way you bypass the IVR and key presses for the working hours. Tell me if you need any more help.
(I edited to include the working hours as condition for the rule).

As a clarification for you, the Custom rule will override whatever you have in the Company Call Handling > Business hours. So it is irrelevant what you have there. Just select a random IVR menu, since the position can't be left blank.