29.1. Improvements from past Audio Libraries

Following is a discussion of the changes/additions from past audio libraries.

29.1.1 Players

There are 2 types of players in the audio library. One is a sequence player which performs the MIDI sequences, and the other is a sound player which replays the sound effects. In the previous audio library, the application could have any number of players, regardless of types.

In n_audio library, the game application is limited to having 2 sequence players and 1 sound player active (registered in the synthesizer driver) at any one time. Therefore, audio processing is simplified and sped up by changing the process from indirect reference to direct reference.

29.1.2 The Envelope

Past audio libraries used an exponential function for the envelope. This was changed to a linear function in n_audio, thereby minimizing the processing required by the audio driver, and speeding up audio processing.

Figure 29-1 Changes to the Envelope

29.1.3 Fixing the Minimum Unit of the Audio Process

In the former audio library, the RSP could create the maximum number (160) of sample data at one time. (We will call the unit that the RSP processes at one time (160 samples) "1 sub-frame".) However, as shown in the Example 29-1, the audio command list is split and the RSP cannot create the maximum number (160) of sample data at one time. Therefore, the whole processing rate deteriorates.

Example 29-1 Partition of the Command List in past Audio Libraries

When the envelope change (Attack -> Decay) occurs within 1 sub-frame, this sub-frame is split into "the remainder of Attack" and "Decay" as shown in the figure above.

Therefore, in n_audio, we set up the minimum unit used by the audio process so that the audio command list is not split. Also, we fixed this minimum unit to the maximum number of sample data that RSP can process at one time.

If the maximum number of sample data is 160 (the same number as past libraries), the envelope change of Example 29-1 is processed as in Example 29-2.

Example 29-2 The Process for Partition of the Command List in the n_audio Library

If the envelope change (Attack - Decay) occurs within 1 sub-frame, the transition point is forced to move to the next complete sub-frame.

Therefore, the audio command list is not split and processing for the CPU and RSP are sped up.

In this way, we have set the minimum unit for the audio process and fixed this value in n_audio.

29.1.4 Processing of Every 184 Samples

When each player is used, the most common playback frequency is 32kHz.

When the playback frequency is 32kHz, the required number of sample data for 1 frame becomes

32000 / 60 = 533.33333 = 533

samples using past libraries. The maximum number of RSP sample data is 160 using past libraries. So, 533 samples are split into 4 sub-frames as illustrated in the Figure 29-2.

Figure 29-2 Partition of the Frame with past Audio Libraries

In this case, the last sub-frame needs to create only 53 samples.

On the other hand, in n_audio, we fixed the minimum unit of the audio process to the maximum number of sample data that the RSP could process at one time. Therefore, we must set the data number created in 1 frame to a multiple integer of the maximum data number of the RSP in n_audio. If the maximum data number is 160 (which is the same number as past libraries), n_audio must create

160 * 4 = 640

samples for 1 frame, as illustrated in Figure 29-3.

Figure 29-3 Partition of the Frame in the n_audio Library (A)

At this moment, the sample data

160 - 53 = 107

is carried over to the next frame as the surplus sample data of this frame. To adjust for this surplus data, you would prepare frames as in Figure 29-4.

Figure 29-4 Partition of the Frame in the n_audio Library (B)

If you use Frame A and Frame B as in Figure 29-5, as time passes it is possible to adjust the previous surplus data.

Figure 29-5 Adjustment of the Surplus Sample Data in the n_audio Library

The total output sample data of these 3 frames is

160 * 4 + 160 * 3 + 160 * 3 = 1600

The actual required number of sample data among 3 frames is

533 * 3 = 1599

Thus, with this calculation it becomes possible to adjust the surplus data of the entire application by using the previous 3 frames repeatedly. For example, we can make a table of the time frame, the number of samples, and the number of surplus samples shown in Table 29-1.

Table 29-1 Used Frames and Surplus Sample Data in the n_audio Library

At time "160" in the table above, the application uses frame A if following the original order. However, because 1 surplus sample is produced by each frame A - frame B - frame B sequence, it uses frame B at time "160" to use the accumulated surplus samples.

This adjustment for the number of created data must be provided on the application side. Also, Table 29-1 has been created using theoretical data. The actual adjustment for surplus data does not always work exactly as represented here.

In Frame A above, you must create 107 more data than the original required number of sample data for 1 frame. Therefore, the processing time for Frame A increases. Though it depends on each case, it is possible for Frame A only, that the increased processing time due to the required processing data, will exceed the processing time allowed by "n_audio."

This problem is avoided as follows.

The maximum number of sample data that the RSP can process at one time is 178.

(32000 / 60) / 3 = 177.77777 = 178

Using this value, the number of surplus data which is produced at each frame is minimized. However, the maximum number of sample data needs to be a multiple of 8 for convenience of the RSP process. Therefore, we increased the maximum number of sample data from 160 to 184 in n_audio.

At this time, the previous frame A becomes like Figure 29-6.

Figure 29-6 Partition of the Frame in the n_audio Library (A) - In the Case of 184

The number of output data in Frame A becomes:

184 * 3 = 552

Thus, in Frame A:

552  - 533 = 19

surplus samples are produced for 1 frame. To adjust these surplus samples for example, we prepare Frame B like Figure 29-7.

Figure 29-7 Partition of the Frame in the n_audio Library (B) - in the Case of 184

Addtionally, it is possible to adjust the surplus data by combining Frame A and Frame B as time passes like Figure 29-8.

Figure 29-8 Adjustment of Surplus Sample Data in the n_audio Library - in the Case of 184

The total of output sample data of these 10 frames is

184 * 3 * 9 + 184 * 2 = 5336

and the actual number of required sample data among 10 frames is

533 * 10 = 5330

Thus, it becomes possible to adjust the surplus data of the whole application by using the 10 frames above repeatedly (see Figure 29-8).

Please refer to the sample program (playseq.naudio) for an example of the method used to adjust surplus sample data. In the sample program, the application decides the frame to use dynamically, so it uses Frame B if the number of surplus sample data becomes the data number for 1 sub-frame (184) or more.

In this way, we changed the maximum number of sample data that the RSP can process at one time from 160 to 184 in n_audio, so that the minimum unit of the audio process is effective at a frequency of 32kHz.