HTML TADS Sounds and Music

HTML TADS provides an extension to HTML that allows a game author to use sound effects and music within a game. Standard HTML doesn't have any support for sound, and the popular browsers mostly delegate sound support to third-party "plug-in" products, so there are no good examples of HTML sound extensions that TADS could have used as a basis. Because of this, the sound features in HTML TADS aren't based on any existing HTML implementations.

Sound and Music Formats

HTML TADS supports several sound formats as of version 2.5.6: MIDI for music (in the standard MIDI file format), Windows WAV (waveform digitized audio) for digitized sound effects; MP3 for compressed, digitized sound and music; and Ogg Vorbis digital audio. MP3 support was added in version 2.2.7; previous versions only supported the MIDI and WAV formats. Ogg Vorbis support was added in version 2.5.6. (The technical name for MP3 is MPEG 2.0 Audio Layer III. HTML TADS supports this as well as Audio Layer II.)

While these formats aren't universal standards, they are widely supported and well-documented. Many multimedia authoring tools will produce files in these formats, and a large number of public domain files are available for all of the formats. In addition, together, these formats address most of the needs that game authors are likely to have when designing games. WAV provides high-quality digitized recordings that can reproduce essentially any sound, while MIDI provides a versatile and compact format for storing music. MP3 and Ogg Vorbis are compressed digital audio formats that offer high fidelity but much smaller file sizes than uncompressed WAV; both work well for sound effects and music.

Why you should use Ogg Vorbis instead of MP3

We recommend using Ogg Vorbis instead of MP3 for digitized audio in your TADS creations.

Ogg Vorbis and MP3 are equivalent in terms of form and function. They both take digitized audio files and compress them to a fraction of their original size. Both are "lossy": they achieve high compression ratios by discarding details in the original signal that they deem inaudible or minimally audible. Both formats have encoders that let you select just how lossy the compression is, as a trade-off against file size. In terms of TADS playback capabilities, Ogg and MP3 are completely interchangeable.

If they're so equivalent, why use Ogg Vorbis instead of MP3, when MP3 is so ubiquitous?

The big reason to use Ogg Vorbis is that it's a fully free and open source format. MP3 is not; certain commercial entities claim patent rights to various aspects of MP3. The patent entanglements make MP3 a pariah on Linux systems, to such a degree that some Linux TADS distros intentionally disable MP3 support. If you use MP3 in your game, players on those Linux systems will encounter dead air on playback. Ogg Vorbis is supported on every Multimedia TADS platform, and it's built in to all of them - users don't have to fuss with codecs or other add-ons.

What's more, Ogg is considered to be the better format technically. It achieves higher compression ratios with better audio quality. Your music and sound effects will most likely sound better if you use Ogg.

Really, the only point in MP3's favor is inertia - you're probably more familiar with it than Ogg, and you probably have an MP3 encoder that you already know how to use. Inertia's hard to overcome, but the encoder problem is easily solved, at least. Search for oggDropXPd for a free Ogg encoder for Windows that you can download and install in less than a minute. Just drop your WAV files on it, and they'll turn into Ogg files.

MIDI

HTML TADS has support for MIDI, but we don't strongly recommend using it for new work. MIDI support on PCs has always been a little unpredictable, because operating systems tend to farm out MIDI playback to the sound card hardware. That makes the quality of MIDI playback highly variable. It also creates limitations in what TADS can do with MIDI playback. For example, TADS can't always perform fades on MIDI tracks, and on many machines it's limited to a single MIDI track at a time, so you can't always cross-fade or overlap MIDI tracks.

MIDI resources in an HTML TADS game must be named with a ".mid" or ".midi" suffix.

MP3 Audio

MP3 is a compressed digital audio format that's suitable for any type of audio, from music to voice recordings to sound effects.

As we mentioned above, we recommend using Ogg Vorbis over MP3, due to the wider availability of Ogg and its superior audio quality and compression power.

MP3 is more formally known as MPEG 2.0 Audio layer III. TADS supports MPEG 2.0 audio layers II and III.

MP3 resources in an HTML TADS game must be named with one of the following suffixes:

    .MPG
    .MP2
    .MP3

The different suffixes are usually used to indicate which sub-format of MPEG the file uses, but for TADS resources, you can use them interchangeably. TADS detects the MPEG sub-format automatically by inspecting the file contents.

Digitized audio is suitable for both music and sound effects, so HTML TADS allows MP3 audio to be used in any layer. TADS doesn't assume anything about which layer is appropriate for a particular MP3, so a <SOUND> tag with an MP3 source must specify a layer.

The Ogg Vorbis Audio Format

Ogg Vorbis is another compressed digital audio format, suitable for digitized sound effects and music. Like MP3, it's a perceptually coded, "lossy" audio compression format. Unlike MP3, it was created as a fully free and open-source format. Ogg is widely considered to have better compression ratios and audio quality than MP3.

Freeware tools for encoding and playing Ogg Vorbis files are available on the Web. You can find information and links to Ogg Vorbis software and sample music files at Vorbis.com. Reference information about the Ogg Vorbis format is available at Xiph.org.

In HTML TADS, Ogg Vorbis resources must be named with the suffix ".OGG".

Ogg Vorbis audio is suitable for both music and sound effects, so the HTML TADS sound system allows this format to be used in any layer. HTML TADS doesn't make assumptions about which layer is appropriate for a particular Ogg Vorbis sound, so all <SOUND> tags with Ogg sources must specify a layer.

Sound Architecture

Layers

Sounds in HTML TADS are divided into four "layers": the background layer, the background ambient layer, the ambient layer, and the foreground layer. (Note: these layers don't have anything to do with MPEG layers, which are simply different encoding formats - see above.)

Each layer is essentially an independent audio output. You can think of the system as an audio mixing studio where you have four CD players hooked up to one set of speakers, with all the outputs mixed together. You can pop a different CD into each player, and start and stop them independently. Each layer corresponds to a CD player, so you can play a different sound file in each layer, and you can control starts, stops, and track changes in each layer individually.

The background layer is used for music. Background music can repeat to provide a continuous soundtrack, and you can program multiple selections that play one after another. You can change the background music whenever you want; for example, each room can have its own background music.

The bgambient (background ambient) layer is used for continuous digitized sound effects. The purpose of this layer is essentially the same as that of the background layer, but the bgambient layer is used for digitized sound effects instead of music.

The ambient layer is used for sound effects that occur in the background, but not continuously like the background music. You can set up a group of ambient sounds that will play randomly to enhance the atmosphere. As with background music, you can change the ambient sounds whenever you want; you can, for example, provide an appropriate set of ambient sound effects for each location.

The foreground layer is used for sound effects that play immediately in response to an event in the game. For example, you could play a sound effect in the foreground when the player opens a door in the game.

For the sake of completeness, we should mention that the sound system was designed with the idea in mind that some operating systems might not be able to handle playback of four tracks at the same time. Currently, all of the actual versions of HTML TADS (the Windows version, and HyperTADS on the Mac) easily handle full simultaneous playback, so this limitation doesn't exist in practice today. If HTML TADS were ever ported to a less capable system, though - and this is unlikely - here's what would happen: sounds in the "front" layer(s) would temporarily preempt sounds in the "rear" layers, to the extent necessary to stay within the system's limits for simultaneous playback. The ordering of the layers from front to back is foreground, ambient, background ambient, and background. Here's an example. Suppose that you're running on a system that can only play back one sound at a time. You have some music playing in the background layer, and somewhere in the middle of the music track, you play a short sound effect (a creaky door opening, say) in the foreground layer. Because of the OS limit of one sound at a time, playing the creaky door effect would momentarily interrupt the background music, and then start it up again where it left off as soon as the creaky door ends. We emphasize that in practice, you should never have to worry about this, since there are no existing ports with this limit and we don't foresee porting to any such systems in the future.

Queueing

Sound is an inherently time-based medium. Text adventures, in contrast, are essentially static, except to the extent that the game state evolves in bursts in response to commands from the player. To bridge this gap, HTML TADS provides automatic management of the time domain. You simply specify which sounds are to be played and in what order, and HTML TADS takes care of the timing details.

At any given time, HTML TADS allows one active sound in each layer. The background sound must be MIDI, MPEG, or Ogg Vorbis and the bgambient, ambient, and foreground sounds must be WAV, MPEG, or Ogg Vorbis.

Within a layer, sounds are queued. This means that sounds play sequentially, in the order that you specify the sounds. When you start a sound, you specify how it interacts with other queued sounds and how it plays:

The <SOUND> tag

The sound features of HTML TADS are activated using a new tag, <SOUND>. This new tag is not an element of the standard HTML definition; this is a TADS extension.

Attributes of the SOUND tag specify the action that the SOUND tag should perform.

The SRC attribute specifies the name of the resource containing the sound data. The resource name must end in .WAV for a wave file; .MID or .MIDI for a MIDI file; .MPG, .MP2, or .MP3 for an MPEG audio file; and .OGG for an Ogg Vorbis file. Refer to Using Resources in HTML TADS for information on how to include resource files in your game.

The LAYER attribute specifies the layer containing the sound. The value of the LAYER attribute must be FOREGROUND, BGAMBIENT, AMBIENT, or BACKGROUND. Although you should always specify a layer for each sound, HTML TADS can sometimes infer the correct LAYER attribute for a sound based on its other attributes:

The RANDOM attribute lets you specify the probability of playing a sound in the ambient layer. Usually, you will want a sound in the ambient layer to be played randomly from time to time; this attribute specifies the probability that the sound will be played at any given time. This attribute takes a value from 1 to 100. A low value makes the sound play infrequently; a value of 100 causes the sound to play at every opportunity (i.e., whenever another sound in the same layer isn't playing).

The REPEAT attribute lets you specify how many times a sound should play. This can be used for all layers. This attribute takes a numeric value specifying the number of times to play the sound, or a value of LOOP to indicate that the sound plays repeatedly until cancelled.

If you don't specify a REPEAT attribute for an ambient sound, and the sound has a RANDOM setting, the default REPEAT=LOOP will be used. If you do specify a REPEAT parameter for an ambient sound, the sound will be played a maximum of the given number of times; the sound will still play only when the system randomly chooses the play it, the probability of which is controlled through the RANDOM attribute.

If you specify a REPEAT attribute for a foreground sound, the sound will be played the given number of times in sequence.

If you specify a REPEAT attribute for a sound in the background or bgambient layers, the sound will be played the given number of times. The actual sequence in which the sound is played depends on the SEQUENCE attribute used in the sound and other background sounds.

The SEQUENCE attribute is used to control the order of repetition for a group of background sounds; it's meant primarily for use with the background and bgambient layers. The SEQUENCE value can be REPLACE, RANDOM, or CYCLE. REPLACE causes the sound to remove any previous background sounds from the queue when the new sound starts. RANDOM leaves any previous sounds in the queue, assuming they have enough REPEAT cycles remaining; when the new sound is finished, if no other background sounds are waiting to be played, the system randomly picks one of the sounds remaining in the queue and plays it again. CYCLE is similar to random, but goes back to the first sound in the queue when the new sound is done.

You can use SEQUENCE to provide transitions with the music. For example, you could specify REPEAT=1 for introductory music, then specify REPEAT=LOOP and SEQUENCE=RANDOM for a set of ongoing background music. By using SEQUENCE=RANDOM, you can prevent the music from becoming too repetitive by randomly varying the order of the tracks.

The INTERRUPT attribute (which takes no value) specifies that the sound is to abort any currently playing sound in the layer and start the new sound immediately. You should usually only use INTERRUPT with foreground sounds.

The CANCEL attribute turns off all currently queued sounds. With no value, a <SOUND CANCEL> tag cancels all queued sounds in all layers. You could use this, for example, when switching to a new room, to turn off all sounds. You can also specify a layer with CANCEL to specify that sounds in that particular layer are to be cancelled; the layer values FOREGROUND, AMBIENT, and BACKGROUND are allowed. For example: <SOUND CANCEL=AMBIENT> cancels only the sounds in the ambient layer.

The ALT attribute lets you specify a textual description of the sound. This could be used by a version of the interpreter that doesn't support sound to display a description of the sound (the Windows version of the interpreter currently ignores this attribute).

The FADEIN and FADEOUT attributes let you fade the track in at the beginning and/or fade it out at the end, as described in detail below. The duration of the fade, in seconds, is given as the value of the attribute. The duration can have a fractional part: for example, FADEIN=2.5 means that the track fades in over two and a half seconds.

When a sound is repeated with the REPEAT attribute, the fade-in applies only to the first iteration, and the fade-out applies only to the last iteration. The main use for REPEAT is to construct ambient background music or atmospheric effects that loop indefinitely, and for that you want the playback to be continuous, without gaps between loop iterations. If you're using a looping track where you specifically want a fade gap between repeats, you can simply prepare the track itself to include fading at each end as part of the actual digitized music or sound, and omit the FADEIN and FADEOUT attributes in the HTML playback.

The VOLUME attribute lets you set the relative playback volume for a sound. The value is an integer ranging from 0 to 100, where 0 is complete silence, 100 is the full, unattenuated level as recorded in the sound file, and values in between are proportionally reduced volume, akin to using a speaker volume dial. (Values outside the 0-100 range are invalid.) VOLUME only affects the track it applies to: it doesn't change the physical speaker volume level of the computer, it doesn't affect any other tracks that might be playing concurrently in the game, and it doesn't persist for subsequent tracks played in the same layer. The purpose of VOLUME is to let you control the relative loudness of sounds when you're playing back multiple tracks at the same time, to achieve the mixing effect you intend.

VOLUME and FADEIN/FADEOUT work together in the obvious way. The VOLUME setting is used as the "base" volume for the main part of the track's playback, so a fade-in gradually increases the loudness from silence to this base level, and a fade-out gradually decreases the loudness from the base level to silence. So, the VOLUME setting is essentially a "master volume" for the track, and the fades are applied relative to this overall base level.

Note that not all HTML TADS interpreters support audio fades or volume levels; interpreters that don't will simply ignore these attributes.

Fades

An audio "fade" is a gradual change of playback volume from silence to full volume (a "fade in"), or from full volume to silence (a "fade out"), usually over an interval of a couple of seconds. Fades are useful for smoothing out transitions between tracks - one track fades out, and then the next track fades in.

A fade only affects the track it applies to. Fades don't alter the audio volume for the whole computer or for the whole game - they just control the volume for one track at a time. If you fade in a track in the foreground layer, for example, the volume of a track playing concurrently in the background layer won't be affected.

The Windows version of HTML TADS supports audio fades, starting with release HT-21. Older Windows versions (and some versions on other platforms) can't do fades, but fade attributes won't do any harm even so - they'll just be ignored if a user plays your game with an older version that doesn't support fades. The systemInfo() function (available in both TADS 2 and TADS 3) lets you check the interpreter capabilities to see if fading is supported.

HTML TADS provides five basic fading modes, all controlled through the FADEIN and FADEOUT attributes of the <SOUND> tag.

1. Fade in at the start of a track. When you start a new sound, you can make it fade in gradually (from silence to full volume) as it starts playing, by including the FADEIN attribute in the <SOUND> tag for the new track. Specify FADEIN=seconds, giving the duration in seconds of the fade-in. As the track starts playing, the volume will be gradually increased over the period you specify. Here's an example:

<SOUND SRC="song.mp3" FADEIN=1.75>

This makes the track fade in over 1.75 seconds. Note that you can use a decimal point to specify a fractional number of seconds.

2. Fade out at the end of a track. When you start a new sound, you can also tell it to fade out gradually (from full volume to silence) just before it finishes playing, by including the FADEOUT attribute.

<SOUND SRC="song.mp3" FADEOUT=2.5>

This says that the track should start fading out at two and a half seconds before the end of the track, so that it's down to zero volume just as it ends.

You can combine FADEIN and FADEOUT, if you want the track to fade in at the beginning and fade out as it finishes:

<SOUND SRC="song.mp3" FADEIN=1.75 FADEOUT=2.5>

The durations of the two fades don't have to be the same.

Note that the FADEOUT setting applies only when the track plays all the way to the end. If you use <SOUND CANCEL> or <SOUND INTERRUPT> to stop the track before it plays all the way through, the fade-out mode of the new tag (the CANCEL or INTERRUPT) will apply instead.

3. Cross-fade between two tracks. As we've just seen, you can smooth out the transition between tracks by fading out at the end of the old track and fading in at the start of the new one. Sometimes you might prefer an even smoother transition - one that doesn't have an abrupt cut-off in one track and jump to another, but one that also doesn't have a fade-out/fade-in gap. One way to do this is with a "cross-fade." This is an effect where the old sound fades out while the new sound simultaneously fades in. Because the two fades overlap, and ramp their volumes in opposite directions, the apparent volume level stays about the same throughout the transition. This means there's no obvious fade, but also no abrupt end to one sound or start to the other; the first track just subtly flows into the second. Audio cross-fades are often used in movies at scene transitions, and they can be useful in similar situations in TADS games.

To create a cross-fade in HTML TADS, you can qualify the FADEIN or FADEOUT attributes with the CROSSFADE keyword. Rather than saying FADEIN=time, you write FADEIN=CROSSFADE,time. For example:

<SOUND SRC="song.mp3" FADEIN=CROSSFADE,1.5 FADEOUT=CROSSFADE,1.5>

When you specify FADEIN=CROSSFADE,time, the new track will cross-fade with the preceding track as the preceding track ends. This will automatically add a fade-out to the old track of the same duration as the fade-in for the new track - it's effectively as though you had specified FADEOUT=CROSSFADE,time on the old track, even if you didn't actually do so.

When you include FADEOUT=CROSSFADE,time, the current track (i.e, the one with the FADEOUT=CROSSFADE) will cross-fade with the next track as the current track ends. The current track will start fading out at time seconds before its normal end, and the next track will start playing immediately, overlapping the fade-out of the old track. The next track simply starts normally - it is not automatically faded. If you want the next track to fade in during the cross-fade period, you have to explicitly say so with a FADEIN attribute in its <SOUND> tag.

Note that FADEIN cross-fades won't work with random sounds. The reason is that, in random mode, the system waits until it's actually time to play the next track to choose which track to play. At that point, if the new track has a FADEIN=CROSSFADE setting, it's too late to apply it, since the previous track has already finished. The correct way to handle cross-fades with random sounds is to use FADEOUT cross-fades instead - these work even in random mode because they apply at the end of a track, after the random selection has been made.

Similarly, FADEIN cross-fades won't work with the last track in a loop (with CYCLE). Even though it would seem like the system should have enough notice in this case, it doesn't: there's no way to know for sure that you won't add another track just before the last track in the queue finishes, so the system can't be sure you want to apply the cross-fade to the first queued track when looping back. Again, the correct approach is to use FADEOUT cross-fades for this situation.

4. Cancel a track in mid-play, with fade-out. If you want to interrupt a track before it finishes, you use the CANCEL attribute as described earlier. Normally, CANCEL just abruptly cuts off the playing track. This can be jarring to a listener, especially for something like ongoing background music. It's often better to use a fade-out on the old sound. You can do this by combining the FADEOUT attribute with CANCEL:

<SOUND CANCEL=BACKGROUND FADEOUT=1.5>

This tells TADS to immediately start fading out the sound currently playing in the background layer, reducing it to zero volume over the next second and a half. When the fade is finished, the sound is stopped, just as with a normal CANCEL, meaning that a new sound (specified with a new <SOUND> tag) as soon as the fade-out finishes. Note that you don't have to wait until the fade-out finishes to write out the <SOUND> tag for the next track - you can write out the new tag immediately, and TADS will put the new sound in the queue, to start as soon as the outgoing fade finishes.

5. Cancel a track in mid-play, with cross-fade. You can use the cross-fade capability when interrupting a track. As we've just seen, the normal behavior when you use CANCEL and FADEOUT together is to start fading the current sound, and wait until the fade finishes to start any subsequent track. However, if you use the CROSSFADE qualifier, the next track will be able to start playing back immediately, while the current track's fade-out proceeds. For example:

<SOUND CANCEL=BACKGROUND FADEOUT=CROSSFADE,1.5>

This causes the current background sound to start fading out, and then allows a subsequent <SOUND> tag to start playback immediately, overlapping the fade-out on the old sound. Note that the new sound must have its own FADEIN attribute if you want a fade-in on the new track; otherwise the new sound will simply start at full volume on top of the fading-out old track.

You can also use <SOUND SRC=file FADEIN=CROSSFADE,time INTERRUPT> to cancel the currently playing track and start a new track, all in one tag. If you use this method, the new track starts playing immediately, with a fade-in of the specified duration, and the old track simultaneously fades out over the same period. (Note that if this tag also has a FADEOUT attribute, that value applies only to the new track; the old track's fade-out duration is still controlled by the new track's FADEIN. The point is to make the overlapping fades of the same duration, so that the volume level stays roughly constant across the transition.)

These two methods (CANCEL plus a new sound tag, and the all-in-one INTERRUPT) have almost identical effects - they both make the old sound start fading out immediately, and they both perform the fade in the background, allowing a new track to be played concurrently while the old one is fading. The second method is a little easier to use because you only need one tag for the whole transition. The first method, however, gives you a little more control, since it lets you specify the fade-out time for the old track separately from the fade-in time for the new track. In fact, you don't even have to specify a fade-in for the new track, which might be desirable if the new track's contents already start with a fade-in.

Fades with REPEAT

The REPEAT attribute is designed for creating ambient background music or atmospheric effects, by combining one or more segments into a loop that plays indefinitely. Since the point is to create an ongoing background track, we don't want any obvious gaps between the segments. As a result, we don't want any fades between segments.

To accomplish this, TADS only applies the FADEIN attribute to the first repetition of a repeated sound, and only applies FADEOUT to the last repetition.

If you're using a group of tracks with SEQUENCE=LOOP or SEQUENCE=RANDOM, here's how fades are handled. FADEIN gets no special treatment, but it doesn't really need it: you should simply be sure not to use FADEIN for any tracks other than the first track of the group. FADEIN will still only apply to the first iteration of the first track, just as for a single repeated track, so the first track won't fade in again when playback loops back (with SEQUENCE=LOOP) or selects the first track again at random (with SEQUENCE=RANDOM). For FADEOUT, TADS looks at the queue just before starting each sound to see if there are any other sounds pending. If so, it ignores the FADEOUT parameter of the new track, since one of the other tracks will presumably start playing as soon as the new track ends.

If you actually want fades between adjacent segments of repeated tracks, you can't get this effect using FADEIN and FADEOUT, but there's a simple trick you can use to work around this. Just edit your audio files so that the tracks themselves fade in at the beginning and fade out at the end.

Sound in Banner Windows

<SOUND> tags are only allowed to appear in the main game window. They are allowed to appear within <BANNER> tags, but note that sounds played from within <BANNER> tags are still effectively part of the main window, and must coordinate with the main window for timing and layer sharing.

<SOUND> tags may not appear in TADS 3 banner API windows. Any sound tags that are displayed in banner API windows will simply be ignored.

Clearing the Screen

Because the <SOUND> tag is part of the formatting information that makes up the display window, sounds are conceptually part of the page on which they appear. This means that all sounds are cancelled as soon as you clear the screen using the clearscreen() function.

If you want to perform full-screen animation or other effects that require erasing the display, but you want your sounds to continue playing, you must use a banner for the animation. A banner lets you replace the information displayed on part of the screen without clearing the entire window. Note that you can use a banner that covers the entire display window (by using the HEIGHT=100% attribute) if you want to use the entire window for your animated effects.

Portability

At the time of this writing, the MIDI, WAV, and MPEG audio formats are supported on both the 32-bit Windows and the Macintosh versions of HTML TADS. Ogg Vorbis is currently supported only on Windows, although Macintosh support is expected soon.

If and when other HTML TADS ports appear, it is likely that some of the audio features will be supported, but it is possible that not all of the formats will be available. Sound requires specialized hardware; not all computers have any sound capabilities, although most reasonably modern machines do. The sound resource formats that HTML TADS supports are not universal standards, but they are very widely used de facto standards and hence are widely supported. It is possible, though, that some future ports will not have access to support libraries for all of the audio formats.

As a game author, you should consider the trade-offs in using sound in your game. If you want to use sound, you should keep in mind that some players will be using systems that don't provide sound support, and some will choose to play with sounds turned off.

Fortunately, in an adventure game, it's relatively easy to use sound as an enhancement rather than as a crucial feature of the game. One of the primary benefits of using sound in a game is that it can add to the atmosphere and mood of the game; even though this type of sound can add a lot to the game, it can be be omitted without ruining the game, since the game will still be fully playable.

Unless you don't mind limiting your audience, you should be careful to avoid using sound effects that provide crucial information. Whenever you use a sound effect as a clue, you should add some textual description of the sound as well, so that the game is still playable without sound.