Binaural Wind Volumes

Recently, we've added an impressive chunk of code to make Vilous: Soldier of the North seem more realistic. We've created a new blueprint actor class that can be modified on an instanced basis to suit the design needs of the level builders. This actor we created is called a "Binaural Wind Volume."

What does the word "binaural" mean?

Binaural is a term that refers to using two ears. In a recording environment, binaural recording is a method using two different audio channels to record something, and then playing the recording back to the listener by means of those independent channels; one channel is assigned to the listener's left ear, and the other the right. This creates a stereo sound environment.

We started off making this system by generating Brownian noise, modifying the sound samples we created to get a more desirable wind sound, and then assigning the samples to left and right audio channels. If you were to listen to the left audio sample file, you'd only hear a wind sound in your left ear and nothing from your right ear.

What does the system do?

The system is meant to better simulate headwind for the player by adding an adaptive audio component to their game. When the player is looking into the direction of the wind, they will hear a sound that sounds like wind rushing into their ears. If the player turns around so that they aren't facing the wind, they'll still hear the sound of wind rushing past their head, but not into their ears; this makes the wind sound quieter.

It would sound a little strange if the falloff of sound was linear and applied to both ears, so this is where the binaural capabilities of the coding come into action. We aren't simulating and checking the direction the player is facing when we determine how loud to make the wind sound as much as we are checking what direction the player's ears are facing. This allows us to independently change the left and right audio tracks based on where the player is looking. That means if a player facing into the wind turn to their right, they'll hear stronger wind sounds in their left ear as opposed to their right ear.

Hearing models:

Since sergals, the race you play as in the game, have a drastically different anatomical setup for perceiving audio, we questioned how we should program our binaural audio receiver checks. A sergal's ears are located at the back of their head, on either side and are slightly cambered out by an angle similar to human ears. Obviously, a human's ears are much more different, being located on the sides of our heads and pointed forwards. Imagine taking the cartilage of your ears and flipping them upside down so your ears are pointed backward; that is similar to how sergal ears should work.

The question comes into play, "how do we let the player hear the wind like this? Do we use a hearing model that is based on how us humans hear sounds for comfort or do we risk disorienting the player by using a sergal based hearing model?"

We recently ran some polls on our Discord server and from Twitter to ask supporters what they'd like to hear in the game. The results were surprising; Discord was split a perfect 50/50 on what hearing model they'd like, and Twitter overwhelmingly preferred that we use the sergal hearing model.

As a professional design choice of the team, we decided that we'd like to hear the wind comfortably with human hearing models. The polls showed that there was enough interest in playing with the alternate hearing model that we added that as a setting for the game; you'll be able to choose what hearing model you'd want to use, but the human hearing model will be enabled by default.

We currently have both hearing models in the game and the player will be able to select how they want to hear the wind from a settings menu.

Features:

Behind the scenes:
Lots of features are at work that are intended to make use of this asset as smooth as possible. Without getting into technical details, these features are set up to reduce the likely-hood of errors and reduce the amount of debugging that may need to occur by presenting problems as soon as possible.

Debug features:
For our level builders, we programmed in a bunch of settings that can be enabled or disabled. When playing the game in the editor, we are able to draw the volume bounds, wind directional arrow, the sprite that represents the actor, and ray traces for other debugging purposes.

Instance based editor tools:
A large number of settings are able to be changed in the editor, per actor. This gives more control to the level builders, as they're able to adjust individual actors as they see fit.

Variable wind intensity:
In addition to being able to set the wind speed on a per-actor basis, wind variation is also able to be set. This is a value that defines limits for the intensity of the wind to randomly jump around between while the sound is playing. When the loudness is adjusted, the pitch of the wind will also be adjusted. Faster wind is louder and more high-pitched, as slower wind is quieter and has more of a bassy rumble to it.


Volumetrically operated:
By having multiple actors in the scene, world builders are able to create complex wind bodies. Individual actors can be placed to make it seem like the wind is blowing through an open window, that it is blowing into the mouth a cave, or that it's taking a specific path through other complex scene geometries. When a player stands inside of the bounds of one of these actors, they'll hear the wind.

Global wind setting:
The Binaural Wind Volume actors don't just work with volumes; we programmed in a setting that, when enabled, will simulate wind everywhere in the world. This setting disables the volume detection so no unneeded processing occurs in the background if the player should happen to overlap the original bounds of the volume.


Directional sensitive binaural audio:
As stated above, we simulated wind intensities per-ear. Looking into the wind will cause a louder sound to be heard whereas looking in the direction the wind is traveling will cause it to be quieter. Looking to the left will cause a louder sound to be heard in the player's right ear, and turning right will cause the same effect in the left ear.


Blocking bodies:
The system is also able to trace for any bodies that may block the wind from getting to the player. This also isn't a linear system as it works on a per-ear basis. A player is able to stand behind something like the wall of a building and still hear the wind just fine, but as they move closer to the wall, they'll begin to enter an envelope where wind isn't rushing past them. The farther they move into this envelope, the quieter the wind will become.

Since this is simulated on a per-ear basis, the player can stand on the left side of the building and only hear wind rushing past their left ear and little blowing past their right ear due to their proximity to the building's wall. Turning their head to the left, exposing their right ear to the wind will allow them to hear the wind in that ear, and since their left ear is now facing out of the wind, they'll hear less wind from that ear.

Dynamic Weather System integration:
The actor is able to act independently or is able to be manipulated by our Dynamic Weather System. The Dynamic Weather System will update the Binaural Wind Volume's heading and wind intensity in relation to direction and speed that the clouds are moving.

Velocity based wind:

As the player moves, the volume of the wind in each ear will update based on the player's velocity in relation to the wind. If the player is facing the wind and begin moving into it, then the wind will get louder. The opposite is true if they're moving with the wind; the wind will get quieter.

If the player enters a still pocket of air, then they will be able to hear a low rumble of the wind as they move forward (or backward "as a sergal") to simulate them walking through the air instead of the air pushing past them. This effect is scaleable based on the current wind intensity, so players running in the direction that the wind is traveling, if the wind is slow, will be able to hear the rumble of "walking wind" more than if there were in a very windy environment. If the wind is strong and the player is running with it, then the overall effect will seem to stop and they may not hear any wind if their speeds match.