NG Design Document/Viewer Architecture/Sound

From Rex community wiki

Introduction

The default audio plug-in for the new viewer would use OpenAL (”Open Audio Library”). OpenAL is a LGPL-licensed open source and cross-platform audio API. OpenAL's API is simple to understand and it provides an easy way to generate audio in 3D space. OpenAL resembles the OpenGL API in style and conventions and therefore it would be a natural addition to the realXtend architecture which already uses the OpenGL rendering API. Many today's AAA games and game engines use OpenAL and it has an active developer community.

Features

OpenAL is foremost a means to generate audio in a simulated 3D space. Thus, legacy audio concepts such as panning and left/right channels are not directly supported. But more importantly OpenAL does include extensions to handle sound-source directivity and distance-related attenuation and Doppler effects, as well as environmental effects such as reflection, obstruction, transmission, and reverberation.

Like OpenGL, the OpenAL core API has no notion of an explicit rendering context, and operates on an implied current OpenAL Context. Unlike the OpenGL specification, the OpenAL specification includes two subsections of the API: the core consisting of the actual OpenAL function calls, and the ALC (”Audio Library Context”) API which is used to manage rendering contexts, resource usage and locking in a cross platform manner. There is also an ALUT (”The OpenAL Utility Toolkit”) library that provides higher level convenience functions meant primarly for non-Windows platforms.

OpenAL has three fundamental primitives or objects: source objects, audio buffers and a single listener. Each object is its own entity and can changed independently. A source object contains a pointer to a buffer, the location, velocity and other attributes of the sound. Buffers contain compressed or uncompressed audio data. There is only one listener (per audio context). The listener attributes are similar to source attributes, but are used to represent where the user is hearing the audio from. All the sources are rendered (mixed and played) relative to the listener. The sound engine performs all necessary calculations as far as distance attenuation, Doppler effect, etc.

Comprehensive list of OpenAL object properties:

• Source
  • AL_PITCH pitch multiplier, always positive
  • AL_GAIN source gain, value should be positive
  • AL_MAX_DISTANCE used with the Inverse Clamped Distance Model to set the distance where there will no longer be any attenuation of the source
  • AL_ROLLOFF_FACTOR the rolloff rate for the source, default is 1.0
  • AL_REFERENCE_DISTANCE the distance under which the volume for the source would normally drop by half (before being influenced by rolloff factor or AL_MAX_DISTANCE)
  • AL_MIN_GAIN the minimum gain for this source
  • AL_MAX_GAIN the maximum gain for this source
  • AL_CONE_OUTER_GAIN the gain when outside the oriented cone
  • AL_CONE_INNER_ANGLE the gain when inside the oriented cone
  • AL_CONE_OUTER_ANGLE outer angle of the sound cone, in degrees, default is 360
  • AL_POSITION X, Y, Z position
  • AL_VELOCITY velocity vector
  • AL_DIRECTION direction vector
  • AL_SOURCE_RELATIVE determines if the positions are relative to the listener, default is AL_FALSE
  • AL_SOURCE_TYPE the source type – AL_UNDETERMINED, AL_STATIC, or AL_STREAMING
  • AL_LOOPING turns looping on (AL_TRUE) or off (AL_FALSE)
  • AL_BUFFER the ID of the attached buffer
  • AL_SOURCE_STATE the state of the source (AL_STOPPED, AL_PLAYING, …)
  • AL_BUFFERS_QUEUED the number of buffers queued on this source
  • AL_BUFFERS_PROCESSED the number of buffers in the queue that have been processed
  • AL_SEC_OFFSET the playback position, expressed in seconds
  • AL_SAMPLE_OFFSET the playback position, expressed in samples
  • AL_BYTE_OFFSET the playback position, expressed in bytes
• Buffer
  • AL_ FREQUENCY frequency of buffer in Hz
  • AL_ BITS bit depth of buffer
  • AL_ CHANNELS number of channels in buffer, > 1 is valid, but buffer won’t be positioned when played
  • AL_ SIZE size of buffer in bytes
  • AL_DATA original location where data was copied from, generally useless, as was probably freed after buffer creation
• Listener
  • AL_GAIN “master gain”, value should be positive
  • AL_POSITION X, Y, Z position
  • AL_VELOCITY velocity vector
  • AL_ORIENTATION orientation expressed as “at” and “up” vectors

Assets

OpenAL has no native file support and therefore every new sound format besides WAV must be decoded separately. Fortunately, this would be a trivial task for certain common file types, such as OGG, using the corresponding open source decoding libraries. MP3 is pretty much ruled out from the supported file types because of the license fees.

Operation

To actually create and hear sounds inside the virtual world, world entities will have sound components exposed by the sound plugin. The sound component would consist from the sound source and the sound buffer. The buffer can be created and deleted after every playback for saving memory or it can be held in memory always for sounds with a high occurrence frequency. The avatar or the camera position would work as the listener. Multiple audio contexts could be created e.g. one for the 3D in-world audio and one for the non-spatial UI sounds.

Citations

1. OpenAL homepage

   (http://connect.creativelabs.com/openal/default.aspx)

2. OpenAL 1.1 Specification and Reference

   (http://connect.creativelabs.com/openal/Documentation/OpenAL%201.1%20Specification.pdf)

3. OpenAL @ Wikipedia

   (http://en.wikipedia.org/wiki/OpenAL)

4. OpenAL Programmer's Guide

   (http://connect.creativelabs.com/openal/Documentation/OpenAL_Programmers_Guide.pdf)