Custom Rule only applies to SOME of Primary Members

I have 8 members in my call queue, but I have a staggered work schedule. so my custom rules are like 8am-10am, 10am-12, 12pm-2pm, 2pm-4pm. Problem is that of the 8 members, only 2 are available to answer phones in the 8am-10am time frame. how do I exclude the other 6?