There are 2 kinds of voicemails.
1- if call doesn't get answered within given time during working hours
2- if somebody calls the call queue during closed hours

To change the behavior for closed hours the path is:
Phone System > Groups > Call queues > click on Call queue > Call Handling & members > After hours

To change the behavior during working hours:
Phone System > Groups > Call queues > click on Call queue > Call Handling & members > Wait settings

Here you need to edit the 4th setting:
When maximum waiting time is reached, send caller to
For example if you want it to disconnect the way to do it is to send callers to an Extension and then select an Announcement only extension. After the announcement is played the call disconnects without leaving a voicemail.

I attached pics to this comment

Great answer @Welcome 01!

Thank's for your answer.

It's the same idea with the custom rules ?

Because i have your parameters but sometimes people leave message on voicemail, i don't now how...

PS : can you show me you parameters for "vociemail and notifications" ?

Always messages on voice mail during openning hours...

Found !

I just miss this in my custom rule !

Unfortunately, my drop down doesn't have the additional choices.

How does one get that?