Welcome to the RingCentral Community

There are 2 kinds of voicemails.
1- if call doesn't get answered within given time during working hours
2- if somebody calls the call queue during closed hours

To change the behavior for closed hours the path is:
Phone System > Groups > Call queues > click on Call queue > Call Handling & members > After hours

To change the behavior during working hours:
Phone System > Groups > Call queues > click on Call queue > Call Handling & members > Wait settings

Here you need to edit the 4th setting:
When maximum waiting time is reached, send caller to
For example if you want it to disconnect the way to do it is to send callers to an Extension and then select an Announcement only extension. After the announcement is played the call disconnects without leaving a voicemail.

I attached pics to this comment