The NSSound volume corresponds to the volume slider that you see in QuickTime Player, iTunes, etc. This is the volume for this particular sound, separate from and, as you have seen, is subject to the system volume.
You can get the current system volume by requesting the audio hardware API. Get the default audio output device, then get its volume. Then you can see if it will be above or below the desired volume level.
(It is worth noting that, in fact, there is not a single global “system volume." There are two default output devices and any number of specific output devices, each with its own volume. The "system volume" usually refers to the default output device volume. .)
Remember, however, that this is not how any other application in the system works, so this behavior will seem unpredictable to the user. Expect error messages in the lines "the volume slider does not work if the volume of the system is above a certain level."
source share