I liked the idea so much that I decided to give it a try on XNA. I also saw a JavaScript implementation online, where the user could write the code directly into an input field, and the application would generate the corresponding sound clip for you. This left me wondering about two points in particular:

1. Can I get the XNA Audio API to work on a Windows Form application as a standalone, i.e. without having a Game instance or even graphics at all?
2. Is there a way to let the user write the code into a text box and hear the result without having to recompile the application?

The answer to both questions is yes. Here’s how I did it.

Using the XNA Audio API as a standalone library

This one was particularly easy to solve. I started by creating a Windows Forms application and manually adding a reference to the Microsoft.Xna.Framework assembly. This was enough to grant me access to everything I needed from XNA, in particular the DynamicSoundEffectInstance class.

There’s one catch though. If you’re not using the Game class (such as in this case), XNA expects you to call the FrameworkDispatcher.Update() method periodically. I’ve read that around 20 times a second is enough so I went with that. You also need to call that method at least once before starting audio playback, or your application will crash.

Here’s how I solved it. Note: If you’re not familiar with XNA’s dynamic audio API then check one of my earlier articles such as this: Creating a Basic Synth in XNA 4.0 - Part II.

1. Created a new thread for the audio.
2. On that new thread, I started by calling FrameworkDispatcher.Update() once.
3. Then I enter a loop where I call FrameworkDispatcher.Update() again, and submit my audio buffers.
4. I added a Thread.Sleep(50) at the end of the loop to make it loop roughly 20 times per second.

Here’s the relevant part of the code. When the form has finished loading, I create my audio buffer, a DynamicSoundEffectInstance object, and start the audio thread:

protected override void OnLoad(EventArgs e)
{
    // Create byte buffer for storing and submitting audio samples
    _buffer = new byte[256*2];

    // Create a new DynamicSoundEffectInstance at 8Khz Mono
    _instance = new DynamicSoundEffectInstance(8000, AudioChannels.Mono);

    // Creates and starts the audio thread
    new Thread(AudioThread).Start();
}

And this is the thread method. The comments should make it self-explanatory:

private void AudioThread()
{
    // Service the XNA Framework once before everything is started
    FrameworkDispatcher.Update();

    // Start the DynamicSoundEffectInstance
    _instance.Play();

    // Go into a loop that repeats around 20 times per second (50ms sleep)
    while (_running)
    {
        // Service the XNA Framework once per loop
        FrameworkDispatcher.Update();

        // Fill and submit buffer
        while (_instance.PendingBufferCount < 3)
            SubmitBuffer();

        Thread.Sleep(50);
    }
}

To make the audio thread quit when the application exists, I did this:

protected override void OnClosed(EventArgs e)
{
    // Triggers the audio thread to close
    _running = false;
}

Finally, here’s what I did to submit the audio buffers. I’ll talk about the GetSample() method later, since it’s linked to the runtime compiler side of the application (the part which lets you write and listen to your own code in real-time). Also notice that in the for-loop I also increment an integer variable called “_time”. This is the value that will correspond to the “t” on our formulas:

private void SubmitBuffer()
{
    // Fill the buffer and advance time
    for (int i = 0; i != _buffer.Length; ++i, ++_time)
        _buffer[i] = (byte) GetSample();

    // Submit buffer
    _instance.SubmitBuffer(_buffer);
}

And just to make it complete, here’s all of the member variables used. Unlike my earlier synth articles, these algorithms work directly with bytes, so I only needed to create a single byte buffer to do the processing:

private DynamicSoundEffectInstance _instance;
private byte[] _buffer;
private int _time;
private Type _generatorType;
private bool _running = true;

That’s all for the audio generation part of the application. Now for the part that takes your one-line algorithm (as a string) and turns it into code that you can execute!

The C# Runtime Compiler

Dynamic languages usually have a method that lets you execute code in the form of a string (frequently named something like “eval”). However, C# does not have such a mechanism. What it does have however, is a way to compile and assemble C# code at runtime. This means that you can write a complete class as a string, feed it through the runtime compiler, and then have access to it like any other class in your project (by making use of some Reflection).

So, imagine we had a class like this:

namespace MusicAlgorithm {
    public static class AudioGenerator {
        public static int Generate(int t)
        {
            return /* YOUR ALGORITHM HERE */;
        }
    }
}

Then all we’d need to do is call the static AudioGenerator.Generate(_time) method using our “_time” variable, and store the resulting value in our audio buffer. Here’s how you can compile and assemble a class like this at runtime. Read through the comments for a better understanding of what I did:

private void SetAlgorithm(string algorithm)
{
    // Reset values
    _time = 0;
    _generatorType = null;

    // Ignore if the user wrote nothing
    if (String.IsNullOrWhiteSpace(algorithm))
        return;

    // Create string containing the code for the class
    // It concatenates the algorithm written by the user with the return statement
    StringBuilder codeBuilder = new StringBuilder();
    codeBuilder.AppendLine("namespace MusicAlgorithm {");
    codeBuilder.AppendLine(" public static class AudioGenerator {");
    codeBuilder.AppendLine(" public static int Generate(int t) { return " + algorithm + "; }");
    codeBuilder.AppendLine(" }");
    codeBuilder.AppendLine("}");
    string code = codeBuilder.ToString();

    // Compile code string in memory. Notice that the language chosen is C#
    // And how the GenerateExecutable and GenerateInMemory properties are set
    CodeDomProvider codeDomProvider = CodeDomProvider.CreateProvider("C#");
    CompilerParameters compileParams = new CompilerParameters { GenerateExecutable = false, GenerateInMemory = true };
    CompilerResults compilerResults = codeDomProvider.CompileAssemblyFromSource(compileParams, new[] {code});

    // On error display message and exit
    if (compilerResults.Errors.HasErrors)
    {
        MessageBox.Show(GetErrorMessage(compilerResults));
        return;
    }

    // Otherwise store a Type reference to the assembly we compiled
    // Using reflection we can call the Generate method from this Type
    _generatorType = compilerResults.CompiledAssembly.GetType("MusicAlgorithm.AudioGenerator");
}

By the way, GetErrorMessage() is just an helper method I made to join all the error messages in a single string for output. It is defined like so:

private static string GetErrorMessage(CompilerResults compilerResults)
{
    StringBuilder errorBuilder = new StringBuilder();
    foreach (CompilerError compilerError in compilerResults.Errors)
        errorBuilder.AppendLine(compilerError.ErrorText);
    return errorBuilder.ToString();
}

By the end of it, if calling SetAlgorithm(algorithm) did not find any errors on your code, then the Type object “_generatorType” will be pointing at the type of the class you compiled at runtime. Using this Type object, you can simply use Reflection to invoke the Generate method on it:

private int GetSample()
{
    // Redirects the call to our compiled code if any
    return _generatorType != null ? (int) _generatorType.GetMethod("Generate").Invoke(null, new object[] {_time}) : 0;
}

And that’s all. I was worried that this might turn out to be too slow for real-time usage, but it seems to work okay, at least on my machine.

Results

Here’s a video of the final result as usual, and the source code:

Source Code

That technique relied on the graphic’s card wrap texture addressing mode to do the wrapping automatically for you, while the spritebatch’s source and destination rectangles were used to to move (changing the source rectangle position) or zoom (change the source rectangle size in relation to destination rectangle) the camera. But it still had the problems that it didn’t support rotation (setting the rotation on SpriteBatch.value would cause the entire quad to rotate, not just the content) or respect the camera’s origin.

In order to do that, we need a bit more control. So this time I’ll show you how to correct this behavior by taking control over the texture coordinate generation process using a simple vertex shader. Using this technique you can take any texture (of any size) and make it repeat infinitely no matter what camera orientation you’re currently in. Consider it an upgrade to the older article!

I’ll start with the video sample for this article which should show you what to expect from it:

Part 1 - Setting up XNA

Okay, so we’d like to load a texture and draw it so that it fills the entire screen and seems to repeat infinitely. Furthermore we want to do this in a single draw call, and also want to be able to transform our camera at will without the effect breaking. You should already know how to do the first two steps of the process from the other article, which are, loading the texture and starting the spritebatch in wrap mode:

1) Load the texture:

// Class scope
Texture2D _bg;

// On LoadContent
_bg = Content.Load<Texture2D>("bg");

2) Activate texture wrapping - Pass SamplerState.LinearWrap to SpriteBatch.Begin:

// Down inside your Draw method
_spriteBatch.Begin(SpriteSortMode.Deferred, null, SamplerState.LinearWrap, [omitted other parameters]);

Next what we will do is draw this texture using a quad (i.e. two triangles connected to each other that form a rectangle - SpriteBatch already does this for us) that ALWAYS covers the entire screen, and leave it to a vertex shader to displace our texture coordinates so that it looks like we’re seeing it from another angle and distance. This means that even though the quad is always standing still, because of our vertex shader, its content will appear to change position, rotation, zoom, etc.

So let’s render that quad…

3) Render the texture as a fullscreen quad - Pass GraphicsDevice.Viewport.Bound as both source and destination rectangles to SpriteBatch.Draw:

// Draw texture to viewport-sized quad setting both source and destination rectangles
_spriteBatch.Draw(_bg, GraphicsDevice.Viewport.Bounds, GraphicsDevice.Viewport.Bounds, Color.White);

Now on to the vertex shader…

Part 2 - Vertex Shader

I’ll be using a vertex shader to take the original texture coordinates for each of the quad’s vertices, and displace them so that we see any entirely different portion of the texture depending on our camera parameters. By doing so, the texture content will appear to move, rotate or zoom, even though the quad’s geometry (the four vertices that make up the quad) will still be fixed to our screen’s corners. Also, it doesn’t matter if the resulting coordinate falls outside the 0.0-1.0 range because we are using the wrap texture addressing mode which automatically brings them back into the 0.0-1.0 range at the end!

Here’s the vetex shader we need to use:

sampler TextureSampler : register(s0);
float2 ViewportSize;
float4x4 ScrollMatrix;

void SpriteVertexShader(inout float4 color : COLOR0, inout float2 texCoord : TEXCOORD0, inout float4 position : POSITION0)
{
    // Half pixel offset for correct texel centering.
    position.xy -= 0.5;

    // Viewport adjustment.
    position.xy = position.xy / ViewportSize;
    position.xy *= float2(2, -2);
    position.xy -= float2(1, -1);

    // Transform our texture coordinates to account for camera
    texCoord = mul(float4(texCoord.xy, 0, 1), ScrollMatrix).xy;
}

technique SpriteBatch
{
    pass
    {
        VertexShader = compile vs_2_0 SpriteVertexShader();
    }
}

The vertex shader is pretty minimalistic. Most of the vertex shader is boilerplate code that is needed for any vertex shader working with SpriteBatch, which I got straight of the MSDN samples. The only relevant line is where I transform the texture coordinates (last line of the vertex shader function) using a matrix that I called the “scroll matrix” (by lack of a better name - I just made that up). That’s the matrix which does all of the work, and I’ll describe it next. You’ll need to supply that matrix along with the viewport’s size from your game.

If you’ve never used a shader before, you should read some other tutorials first, but basically you need to put this code on a text file, give it an “.fx” file extension and add it to your content pipeline. You load it just like a Texture or another content pipeline object, but using the class Effect instead.

Part 3 - Back on XNA: Generating the Scroll Matrix

Creating the “scroll matrix” is not too hard. If you already have a camera class, you only need to add this method to it:

public Matrix GetScrollMatrix(Vector2 textureSize)
{
    return Matrix.CreateTranslation(new Vector3(-Origin / textureSize, 0.0f)) *
        Matrix.CreateScale(1f / Zoom) *
        Matrix.CreateRotationZ(Rotation) *
        Matrix.CreateTranslation(new Vector3(Origin / textureSize, 0.0f)) *
        Matrix.CreateTranslation(new Vector3(Position / textureSize, 0.0f));
}

If you compare it with your camera’s usual View Matrix you’ll notice a few differences. This one is basically a World Matrix (notice the S-R-T order of multiplication instead of T-R-S) with the catch that all translations (i.e. Position and Origin) are mapped to the texture’s space (which is sometimes called uv space too - a two dimensional space where where (0,0) corresponds to the top-left corner of the texture, and and (1,1) corresponds to the bottom-right corner of the texture) before being applied, which can be done by dividing their magnitudes by the texture’s size.

This matrix basically figures out where each of the corners of our screen/quad lie in texture space based on the camera’s current transform. After figuring that out, the default pixel shader takes care of the rest!

Part 4 - Loading and using the Effect

Now the only thing we need to do is load the vertex shader effect, pass it the camera’s “scroll matrix” and viewport size, and use it on our SpriteBatch when drawing the texture:

1) Load effect:

// Global scope
Effect _effect;

// On LoadContent
_effect = Content.Load<Effect>("infinite");
_effect.Parameters["ViewportSize"].SetValue(new Vector2(GraphicsDevice.Viewport.Width, GraphicsDevice.Viewport.Height));

2) Pass scroll matrix to the effect:

// On Draw
// Prepare the effect with a view matrix
_effect.Parameters["ScrollMatrix"].SetValue(_scrollCamera.GetScrollMatrix(new Vector2(_bg.Width, _bg.Height)));
// Start the SpriteBatch using LinearWrap and the scroll effect
_spriteBatch.Begin(SpriteSortMode.Deferred, null, SamplerState.LinearWrap, null, null, _effect);

3) Prepare SpriteBatch to use our effect:

// On Draw
// Start the SpriteBatch using LinearWrap and the scroll effect
_spriteBatch.Begin(SpriteSortMode.Deferred, null, SamplerState.LinearWrap, null, null, _effect);

And that’s all there is to it. Check the source code below for the complete source code for this article:

Final Thoughts

My first version of this article used a pixel shader instead of a vertex shader to do all the work. I’ve now changed it into a vertex shader instead which gives the same result, but is much, much more efficient (e.g. on my sample, with a vertex shader I transform only four texture coordinates for the entire effect, while with a pixel shader I was transforming hundreds of thousands of texture coordinates). For reference, here is the pixel shader I was using.

Warning: Use the vertex shader version. It’s a lot more efficient.

sampler TextureSampler : register(s0);
float4x4 ScrollMatrix;

float4 main(float4 color : COLOR0, float2 texCoord : TEXCOORD0) : COLOR0
{
    // Convert our uv coordinates into a 4d vector for matrix multiplication
    // Transform it by the Scroll Matrix
    float2 coords = mul(float4(texCoord.xy, 0, 1), ScrollMatrix).xy;

    // Sample the texture at the calculated spot
    return tex2D(TextureSampler, coords) * color;
}

technique Infinite
{
    // Make sure you're using Wrap mode and a full screen quad
    pass Pass1
    {
        PixelShader = compile ps_2_0 main();
    }
}

Source Code

Introduction

On the day I started writing my first shaders on RenderMonkey, I had to figure out how to add a skybox to my scene so that I could properly test effects such as environment mapping. I searched around RenderMonkey’s interface but did not find any built-in way of rendering a skybox, so I ended up writing it on my own. I thought there could be a more intuitive way of doing this, so I figure it might be of some worth sharing my experience here. But if there is indeed an easier way to accomplish this, please let me know!

The trick to rendering the skybox was to add a second shader pass to the effect, and using the first one for the skybox while the second one is used for the model. The implementation I’ll be showing here works by rendering a simple texture mapped cube model (generated procedurally by RenderMonkey) which is always centered around the camera. This way, no matter where your camera is, the skybox will always look like it’s staying far away from the it. In order to be able to render the cube from inside out, I had modify the graphic card’s render state and invert the culling order, and I also had to disable writing to the Z-buffer while rendering the skybox, so that it didn’t interfer later with my scene on the second pass. As for the texturing of the skybox, I used one of the default cubemaps and the texCUBE HLSL intrinsic do to all the work. Well, let’s get started!

Starting the Effect

The first step is to create a new DirectX effect on your workspace and add a second pass to it by right-clicking on the effect and choose “Add Pass”:

Your workspace should look like this afterwards:

From now on you’ll be working mostly on the first pass (Pass 0) to set up that skybox. You’ll need to create a cube model, prepare some variables and a cubemap texture for the shader, write the shader itself and configure the graphic card’s render state properly. But don’t worry, I’ll guide you step by step.

Creating the Cube

To create the cube model that will be used as a skybox, right-click on your effect and add a new model, but don’t choose any model from the list. Instead click on the default model which is on top of the list. Rename it to “Box” if you want so that you don’t confuse it with the existing model.

Now right-click on your new model, and select “Geometry Generator” under the “Generator” menu option. This will fill your model with data that is procedurally generated by RenderMonkey based on your choices:

On the dialog that pops up, create a box of arbitrary size. I created a 25x25x25 box which worked fine for me, because depending on your near and far planes are configured, choosing a box that is too small or too large may cause problems:

And before we forget, head inside your pass 0, and right-clicking on the model Reference Node there, assign it to your newly created box, because it’s probably still be pointing at your first model:

Creating Variables

The skybox shader I’ll be showing below makes use of three global variables: the position of the camera, the view-projection matrix and the cubemap texture used for the sky:

float4 ViewPosition;
float4x4 ViewProjection;
sampler Texture;

So go ahead and create each of these variables as explained on the previous tutorial, remembering to choose the appropriate data types and semantics. The ViewProjection matrix should already be created for you by default, I simply renamed it because I don’t like the default notation. As for the ViewPosition, can find it under the Float predefined variables:

For the texture, use the specific Cubemap option to load it:

And don’t forget to also add a Texture Object called “Texture” to your pass 0, linked to the cubemap you just created:

Optionally, you can configure your texture object to use a better filtering scheme:

Adding the Code

Now it’s time to add the vertex and pixel shader code. You can find it ready below, so simply copy it into the vertex shader and pixel shader sections of pass 0. The code is commented to explain what it’s doing, but if you have any questions let me know!

The vertex shader:

float4 ViewPosition;
float4x4 ViewProjection;

struct VS_INPUT
{
    float4 position : POSITION0;
};

struct VS_OUTPUT
{
    float4 position : POSITION0;
    float3 position3D : TEXCOORD0;
};

// Constructs a translation matrix
float4x4 translation(float3 position)
{
    return float4x4(
        float4(1,0,0,0),
        float4(0,1,0,0),
        float4(0,0,1,0),
        float4(position.xyz, 1)
    );
}

VS_OUTPUT vs_main(VS_INPUT input)
{
    VS_OUTPUT output;

    // Create a world matrix that translates the skybox to the camera position
    float4x4 world = translation(ViewPosition.xyz);

    // Combine it with the existing view projection matrix
    float4x4 WorldViewProjection = mul(world, ViewProjection);

    // Transform your vertex into place
    output.position = mul(input.position, WorldViewProjection);

    // Save the original vertex position in model space
    // Since the cube model is centered around the origin, you can use these
    // vertex positions directly as directions for your cubemap :)
    output.position3D = input.position;

    return output;
}

The pixel shader:

sampler Texture;

struct VS_OUTPUT
{
    float4 position : POSITION0;
    float3 position3D : TEXCOORD0;
};

float4 ps_main(VS_OUTPUT input) : COLOR0
{
    // Use the original vertex position to sample the texture cube
    return texCUBE(Texture, input.position3D);
}

Configure Render States

You probably still can’t see anything, but the shader should already compile without errors. The reason you can’t see anything is because you’re seeing the cube model from inside out, and since its faces are facing the wrong direction they aren’t rendered. Let’s correct that now by changing the culling mode on our graphics card. Another thing we need to do is disable writing to the z-buffer because we want the sky to always appear last in the scene.

To add these changes, simply right-click on pass 0 and select “Add Render State Block” from the list. Now repeat the same thing on pass 1 because you’ll want to undo your changes there before rendering your model. Match your settings to the ones on the pictures below.

Pass 0 Render State Options:

Pass 1 Render State Options:

Conclusion

And that’s it! Now you can change over to pass 1 and write another shader there, because the skybox will still be displayed on the background, and react correctly to the camera. Here’s the result:

And having a proper skybox in place opens the door to testing many different effects involving reflection and refraction such as this:

Just one final tip: by default, your pass 1 is referencing the same Stream Mapping structure as pass 0. If you need different sets of data for both passes, just create a new Stream Mapping object on your effect, and using the right mouse button, change the reference on pass 1 to your new instance.

That’s all for today, I hope you enjoyed the article.

I’ll be using HLSL for the scope of this tutorial, but I intend to focus only on how to get RenderMonkey up and running, so I won’t be teaching you how to program in HLSL. If you don’t have any experience, here’s a nice resource for getting started with HLSL - kudos to Catalin Zima for the wonderful tutorial. After learning the syntax, you should also learn how to implement some basic shaders. The XNA education catalogue has a series just for that!

Shader Code

The shader used in this tutorial mixes ambient and diffuse lighting with a diffuse texture map. Here’s the code for the shader:

float4x4 WorldViewProjection;
float4x4 WorldInverseTranspose;
float3 LightDirection;
float4 Ambient;
sampler Texture;

struct VS_INPUT
{
    float4 position : POSITION0;
    float3 normal : NORMAL0;
    float2 texturecoord : TEXCOORD0;
};

struct VS_OUTPUT
{
    float4 position : POSITION0;
    float3 normal : TEXCOORD0;
    float2 texturecoord : TEXCOORD1;
};

VS_OUTPUT vs_main(VS_INPUT input)
{
    VS_OUTPUT output;
    output.position = mul(input.position, WorldViewProjection);
    output.normal = mul(input.normal, WorldInverseTranspose);
    output.texturecoord = input.texturecoord;
    return output;
}

float4 ps_main(VS_OUTPUT input) : COLOR0
{
    float3 normal = normalize(input.normal);
    float3 light = normalize(LightDirection);
    float4 diffuse = tex2D(Texture, input.texturecoord);
    float diffuseContribution = clamp(dot(normal, -light), 0, 1);
    return Ambient + diffuse * diffuseContribution;
}

And what it’s supposed to look like:

So let’s get started!

Creating the Effect

When starting RenderMonkey you’ll get an Empty Workspace that’s ready to be used. Your application should look like this:

A lot of your work will be done on the Workspace Window on the left, which by default shows a single empty workspace. So let’s start by adding a new Effect to it. Right-click on Effect Workspace and add a new Default Effect of your desired type (2). Since I’m programming in HLSL, I created a new DirectX shader. There are many templates ready that you can use when starting your own shaders, but for this tutorial I’ll just choose “DirectX” from the list which creates the most basic shader possible (i.e. a vertex shader that simply applies a WorldViewProjection transformation and a pixel shader that always outputs the same color).

Once you have your Effect created, your window should look like this:

Now we’re ready to start setting everything up!

Changing the Model

You probably noticed that the default model used by RenderMonkey is a sphere. That’s not the best choice for testing your shaders, so let’s start by changing it. On your Workspace Window simply right-click on Model and select a new one:

I’ll use a simple teapot:

Setting Variables

The next step is to create a link between RenderMonkey and the global variables used in our shader. The global variables used by our shader are as follows:

float4x4 WorldViewProjection;
float4x4 WorldInverseTranspose;
float3 LightDirection;
float4 Ambient;
sampler Texture;

We’ll need to create a new variable inside our Effect on the Workspace Window for each global variable in our code, so that RenderMonkey can interact with our shader. Make sure to give them the same name and the appropriate data type too. Creating a new variable is as simple as right-clicking on your Effect, going under Add Variable and picking the right data type from the list:

You can start by creating a Float3 called LightDirection and a Color called Ambient.

As for defining our matrixes (such as WorldViewProjection and WorldInverseTranspose), these usually have special meanings, and we’d like RenderMonkey to establish the right connection between them and our scene (e.g. link the View matrix to our camera). To do that, simply use one of the Predefined matrixes that RenderMonkey provides, as each of them already come with the appropriate semantic assigned:

Finally, for creating the texture variable that will be sampled by the shader, we need to do it in two steps.

First, right-click on your Effect, select Add Texture and load a 2D Texture of your choice:

Next, scroll down to Pass 0, right-click on it and add a new Texture Object, linking it to the texture you just created. Be sure to name this Texture Object the same as the sampler used in your code (in our case, Texture):

I also got rid of the matrix that came with the default effect since I won’t be needing it. When you’re done, your Effect tree should look like:

Binding Shader Inputs

One other step we need to take, that is frequently forgotten, is to bind RenderMonkey to our vertex shader inputs so that the correct data is streamed. Looking back on our code, we had the following vertex shader input structure:

struct VS_INPUT
{
    float4 position : POSITION0;
    float3 normal : NORMAL0;
    float2 texturecoord : TEXCOORD0;
};

We’ll need to mimick this structure on RenderMonkey. Inside your Effect tree, right-click on Stream Mapping and choose Edit:

On the window that appears, simply add two new channels and configure them such that they’re compatible with the data types and semantics of your vertex shader input. In our case:

Create the Code

Now for the most important part, entering the code. Scroll down to Pass 0 and right-click on Vertex Shader, choosing Edit from the list. That should open the Code Editor for your effect, with the default implementation still on it:

The most relevant thing to notice here is that unlike most shaders you write, in RenderMonkey you will write your vertex shader and your pixel shader in separate tabs, and the variables you define in one of the tabs are local to it (i.e. can’t be accessed by the other tab).

This means that you’ll have to split your shader code in two, the vertex half and the pixel half. That’s no big deal though. Here’s what it looks like after the split (refer to the full implementation on top for comparison).

Under the Vertex Shader tab:

float4x4 WorldViewProjection;
float4x4 WorldInverseTranspose;

struct VS_INPUT
{
    float4 position : POSITION0;
    float3 normal : NORMAL0;
    float2 texturecoord : TEXCOORD0;
};

struct VS_OUTPUT
{
    float4 position : POSITION0;
    float3 normal : TEXCOORD0;
    float2 texturecoord : TEXCOORD1;
};

VS_OUTPUT vs_main(VS_INPUT input)
{
    VS_OUTPUT output;
    output.position = mul(input.position, WorldViewProjection);
    output.normal = mul(input.normal, WorldInverseTranspose);
    output.texturecoord = input.texturecoord;
    return output;
}

Under the Pixel Shader tab:

float3 LightDirection;
float4 Ambient;
sampler Texture;

struct VS_OUTPUT
{
    float4 position : POSITION0;
    float3 normal : TEXCOORD0;
    float2 texturecoord : TEXCOORD1;
};

float4 ps_main(VS_OUTPUT input) : COLOR0
{
    float3 normal = normalize(input.normal);
    float3 light = normalize(LightDirection);
    float4 diffuse = tex2D(Texture, input.texturecoord);
    float diffuseContribution = clamp(dot(normal, -light), 0, 1);
    return Ambient + diffuse * diffuseContribution;
}

The only points worth noting is that I repeated my VS_OUTPUT structure since it was needed on both sides, and I cleaned my global variables so that only the ones that were used in each file were present.

Also, ensure that your vertex shader and pixel shader functions names match the ones defined under Entry Point. By default, these should be “vs_main” and “ps_main” respectively.

Now let’s see the code in action! Simply press the Compile button on the toolbar:

And this is what you should get:

It looks all white! But don’t worry, that’s only because your LightDirection and Ambient variables are still with their default values. RenderMonkey provides an interesting way for you to edit your variables, so that you may see your shader updating in real-time as you change them!

Tweaking Variables

There are two variables we’d like to tweak here: LightDirection and Ambient. Start by right-clicking on your LightDirection variable and setting it as an Artist Variable. Colors are set as Artist Variables by default so you don’t need to repeat the process for your Ambient color:

Now fire up the Artist Editor window:

And this is what you’ll see on the right side of your application:

Now all you have to do is tweak the variables to your liking. Expand your options in the Artist Editor window so that you have better control over them. In my case I specified that my LightDirection vector should be normalized, and I set my Ambient color to a very dark color:

Texture Sampler Options

Just to throw in one last tip, the texture on my teapot looks a bit rough. I’m guessing it’s probably being sampled using point filtering. Let’s change that!

Right-click on your Texture Object and choose Edit. That’s where you can configure everything about your sampler! I’ll simply change my magnification/minification/mipmapping filters to linear, which should make the texture appear smoother:

Conclusion

And that’s it! You can now maximize your preview window and enjoy. Oh, and you can use the camera controls located on the toolbar to navigate the scene:

Here’s the final result:

I hope you enjoyed this tutorial, and as usual feel free to post any questions on the comment section below.

I implemented it as a simple optional rectangle property called Limits which restricted camera movement so that the player could not see anything beyond that region.

However, I considered the simplest of the scenarios where you don’t need this functionality AND camera zooming/rotation at the same time. To answer a question posted under that article, this time I’ll be presenting a more elaborate implementation that also takes zoom into consideration.

Camera rotation will still be kept out of the equation, since I can’t imagine how it could be made to work with sensible results considering that the viewport is rectangular.

Introduction

I’ll start by presenting a little video of what the purpose of this article is. You can download the source code at the end as usual:

Now onto how it’s done. Let us consider the following simple Camera interface for our example:

public interface Camera
{
    public Viewport Viewport { get; }
    public Vector2 Position { get; set; }
    public float Zoom { get; set; }
    public Matrix ViewMatrix { get; }
    public Rectangle? Limits { set; }
}

What we’d like to achieve is that whenever the Limits property has a value assigned to it, any changes to Position and Zoom should be validated to ensure the player never sees anything beyond the limited region. Changing the camera’s limit should also revalidate everything.

To make this task easier, we’ll create two helper methods, ValidateZoom() and ValidatePosition() which will be called from within our setters above and correct any invalid values for us. Once we have these two methods implemented, the rest is trivial.

Validate Zoom

Making sure the camera’s zoom is valid is not too hard once you think about it. In general terms what you’d like to achieve is to ensure that the area that the camera can see is never larger than the limiting rectangle.

So, exactly how big is this area that the camera can see? Well, we know that when the camera isn’t zoomed at all (i.e. the Zoom property is 1), then the area the camera can see corresponds exactly to the game’s Viewport. It is also intuitive to think that if you’re zoomed in, you can see less of the world. Conversely, if you’re zoomed out you can see more than you could before.

This implies that the camera “size” (i.e. the size of the area visible by the camera) is inversely proportional to the camera’s zoom, or in mathematical terms:

CameraSize = ViewportSize / Zoom

Now that we know how large the camera is, let us revisit our initial statement - “Ensure that the area that the camera can see is never larger than the limiting rectangle”. We can just as easily model that with a mathematical expression:

CameraSize <= LimitSize

Combining both equations into one and rearranging it we arrive at the following conclusion:

Zoom >= ViewportSize / LimitSize

All that remains is to implement it. The only problem here is that while Zoom is a scalar (float), ViewportSize and LimitSize are both vectors so we can’t compare them directly. The solution is to compare separatedly for the X and Y axes and ignore the smallest value of the two. In code:

private void ValidateZoom()
{
    if (_limits.HasValue)
    {
        float minZoomX = (float)_viewport.Width / _limits.Value.Width;
        float minZoomY = (float)_viewport.Height / _limits.Value.Height;
        _zoom = MathHelper.Max(_zoom, MathHelper.Max(minZoomX, minZoomY));
    }
}

Validate Position

Validating the camera’s position after moving it or changing zoom is a bit harder. First you need to know where the top left corner of the camera stands in World space. For that you need to take your ViewMatrix and invert it so that you can transform points from View space back into World space, and apply it to the camera’s local top left corner (which in View space is simply Vector2.Zero), which gives you its position in world space. That position is then clamped inside the limiting rectangle’s min and max corners. However there are two catches that you need to take care of.

The first one is that, we’re clamping our top LEFT corner against the limiting rectangle’s bottom RIGHT corner, so in order to compensate we need to subtract it the camera’s size too (remember the formula above?). The second one is that the camera’s top left corner in world space doesn’t necessarily correspond to the camera’s position (because of zoom). To solve that I start by taking note of how much both of these values differ in order to correct that offset at the end. Here’s the code:

private void ValidatePosition()
{
    if(_limits.HasValue)
    {
        Vector2 cameraWorldMin = Vector2.Transform(Vector2.Zero, Matrix.Invert(ViewMatrix));
        Vector2 cameraSize = new Vector2(_viewport.Width, _viewport.Height) / _zoom;
        Vector2 limitWorldMin = new Vector2(_limits.Value.Left, _limits.Value.Top);
        Vector2 limitWorldMax = new Vector2(_limits.Value.Right, _limits.Value.Bottom);
        Vector2 positionOffset = _position - cameraWorldMin;
        _position = Vector2.Clamp(cameraWorldMin, limitWorldMin, limitWorldMax - cameraSize) + positionOffset;
    }
}

Updating the Properties

All that remains is to make use of these two methods in order to validate our data. Just add them inside your property setters. When changing position you don’t need to do any zoom validation. When changing zoom however, you have to follow it with a position validation because if you were standing close to the border, the position might end up being invalid after zooming out. Same thing when changing the camera’s limit, don’t forget to validate everything.

public Vector2 Position
{
    get { return _position; }
    set
    {
        _position = value;
        ValidatePosition();
    }
}

public float Zoom
{
    get { return _zoom;	}
    set
    {
        _zoom = value;
        ValidateZoom();
        ValidatePosition();
    }
}

public Rectangle? Limits
{
    set
    {
        _limits = value;
        ValidateZoom();
        ValidatePosition();
    }
}

Usage Example

Using this technique is as simple as setting a new Rectangle to the Limits property, such as:

_camera = new Camera(GraphicsDevice.Viewport);
_camera.Limits = new Rectangle(0, 0, 512, 512);

No matter how big or small your Viewport and Limits rectangle currently are, the camera’s position and zoom are initialized so that the player is guaranteed to only see the inside of that region. As you saw on the video earlier, I cycle through rectangles of many different sizes and locations and the camera is automatically updated to focus on that content.

Source Code

So, here’s the deal! I’ve been working on 2D point-and-click graphic adventure game engine and editor lately, and what I’d like to share today is the general idea of how I ended up implementing the pathfinding for my engine.

If you’ve never seen a 2D point-and-click graphic adventure game, it’s a game genre that used to be really popular during the 90s. I think this video sums it all up (literally):

So, in most point-and-click graphic adventure games, whenever you click on the screen, your character tries to get as close as he can to that location, while evading all the obstacles along the way. But unlike a tile-based game where the walkable areas are easily represented by a regular grid, the floor on a graphic adventure game can have virtually any arbitrary shape.

In the old days (in the golden era of LucasArts and the SCUMM) they would often represent the walkable areas as a set of walk boxes, which were really nothing more than simple horizontal trapezoids laid down to cover the entire floor area. A pre-processing stage would then calculate the shortest path between each and every walk box, and store the results in a matrix, so they could simply be looked up at runtime (which traded an higher memory footprint for increased performance).

Nowadays, computers have become so advanced that it’s not really necessary to do this - we can simply run the pathfinding in realtime and be done with it. And as for the floor representation, I wanted something that would be easier to define in my editor than trying to manually cover the floor with individual trapezoids. I wanted the user to be able to simply draw any shape he wanted, no matter how complex, and have the character automatically know how to navigate it!

So here’s a video of what I came up with:

Algorithm

The pathfinding algorithm used in the video is really just a regular A*. I recommend this resource to learn how to implement it in C#, and what I used is pretty much just a direct translation from the method described there! What’s more special here is how the graph is created, which is what enables us to walk all over the surface of the polygon without ever leaving it, and always finding the shortest route possible. So, here’s the process needed to create the graph:

1. First, create a graph node for each concave vertex of the polygon. If the polygon has holes, then you should also include the convex vertices of the hole polygons.
2. Run a line-of-sight algorithm between each pair of graph nodes and link those which are in direct line-of-sight from each other.

As an example, here’s a simpler polygon than the one in the video. I’ve marked all the concave vertices with a circle, and you can also see the connections between each of them. This is all you need to be able to walk over the polygon!

This sort of graph is usually referred to as a visibility graph, and because of that, the pathfinding method I used is also known as points-of-visibility pathfinding. You can read more about it in the first two Game Programming Gems books, on the articles “The Basics of A* for Path Planning” and “Expanded Geometry for Points-of-Visibility Pathfinding” respectively. The graph could still be optimized to remove a lot of redundant edges that will never be needed to move around obstacles, but I left that for a later stage since I’m not running into any performance issues at the moment.

With your graph structure created, you can apply the A* algorithm over it by following these steps:

1. First verify if both the start and end points of the path are inside the polygon. If the end point is outside the polygon you can optionally clamp it back inside.
2. Then start by checking if both points are in line-of-sight. If they are, there’s no need for pathfinding, just walk there!
3. Otherwise, add the start and end points of your path as new temporary nodes to the graph.
4. Connect them to every other node that they can see on the graph.
5. Run your A* implementation on the graph to get your path. This path is guaranteed to be as direct as possible!
6. Finally, remove the two temporary nodes from the graph.

Taking the graph from earlier, this process could be represented as something like:

Where the red and green dots are the start and end nodes which have been added to the graph, and linked to every other node they can see (the green lines). Then a path is taken from this new graph (the light blue line).

Code Snippets

I don’t have a full sample that I can share at this moment, but I’ll describe two of the most tricky parts of the process. Hopefully you can figure out the rest from the references I gave above.

One essential part of the process is knowing if two points inside a polygon are in line of sight or not. In order to do that, I run a line segment intersection test between the line segment formed by both nodes, and all edges of the polygon. If both nodes are in line-of-sight, then the test should return false. It’s not very efficient since I compare against all edges of the polygon, but it gets the work done. Heres the helper method I use for the line segment intersection test:

public static bool LineSegmentsCross(Vector2 a, Vector2 b, Vector2 c, Vector2 d)
{
    float denominator = ((b.X - a.X) * (d.Y - c.Y)) - ((b.Y - a.Y) * (d.X - c.X));

    if (denominator == 0)
    {
        return false;
    }

    float numerator1 = ((a.Y - c.Y) * (d.X - c.X)) - ((a.X - c.X) * (d.Y - c.Y));

    float numerator2 = ((a.Y - c.Y) * (b.X - a.X)) - ((a.X - c.X) * (b.Y - a.Y));

    if (numerator1 == 0 || numerator2 == 0)
    {
        return false;
    }

    float r = numerator1 / denominator;
    float s = numerator2 / denominator;

    return (r > 0 && r < 1) && (s > 0 && s < 1);
}

The other important point is to know whether a vertex in a polygon is convex or not. I use the following method for this (remember, if the polygon you’re testing is a hole, then you should invert the result):

public static bool IsVertexConcave(IList<Vector2> vertices, int vertex)
{
    Vector2 current = vertices[vertex];
    Vector2 next = vertices[(vertex + 1) % vertices.Count];
    Vector2 previous = vertices[vertex == 0 ? vertices.Count - 1 : vertex - 1];

    Vector2 left = new Vector2(current.X - previous.X, current.Y - previous.Y);
    Vector2 right = new Vector2(next.X - current.X, next.Y - current.Y);

    float cross = (left.X * right.Y) - (left.Y * right.X);

    return cross < 0;
}

To check if a point inside a polygon I used the following method which has some extra code added for tolerance near the edges:

public static bool Inside(IList<Vector2> polygon, Vector2 position, bool toleranceOnOutside = true)
{
    Vector2 point = position;

    const float epsilon = 0.5f;

    bool inside = false;

    // Must have 3 or more edges
    if (polygon.Count < 3) return false;

    Vector2 oldPoint = polygon[polygon.Count - 1];
    float oldSqDist = Vector2.DistanceSquared(oldPoint, point);

    for (int i = 0; i < polygon.Count; i++)
    {
        Vector2 newPoint = polygon[i];
        float newSqDist = Vector2.DistanceSquared(newPoint, point);

        if (oldSqDist + newSqDist + 2.0f * System.Math.Sqrt(oldSqDist * newSqDist) - Vector2.DistanceSquared(newPoint, oldPoint) < epsilon)
            return toleranceOnOutside;

        Vector2 left;
        Vector2 right;
        if (newPoint.X > oldPoint.X)
        {
            left = oldPoint;
            right = newPoint;
        }
        else
        {
            left = newPoint;
            right = oldPoint;
        }

        if (left.X < point.X && point.X <= right.X && (point.Y - left.Y) * (right.X - left.X) < (right.Y - left.Y) * (point.X - left.X))
            inside = !inside;

        oldPoint = newPoint;
        oldSqDist = newSqDist;
    }

    return inside;
}

And to check for line of sight between two points inside the polygon, it was basically this:

bool InLineOfSight(Polygon polygon, Vector2 start, Vector2 end)
{
  // Not in LOS if any of the ends is outside the polygon
  if (!polygon.Inside(start) || !polygon.Inside(end)) return false;

  // In LOS if it's the same start and end location
  if (Vector2.Distance(start, end) < epsilon) return true;

  // Not in LOS if any edge is intersected by the start-end line segment
  foreach (var vertices in polygon) {
    var n = vertices.Count;
    for (int i = 0; j < n; i++)
      if (LineSegmentsCross(start, end, vertices[i], vertices[(i+1)%n]))
        return false;
  }

  // Finally the middle point in the segment determines if in LOS or not
  return polygon.Inside((start + end) / 2f);
}

Finally, you might have noticed how in the video I was joining and subtracting polygons from each other while drawing. This process was made easy due to a library called Clipper (which you can get here). It’s pretty easy to use! What I’m doing is close to:

Clipper clipper = new Clipper();
clipper.AddPolygons(_fullPolygon, PolyType.ptSubject);
clipper.AddPolygon(_temporaryPolygon, PolyType.ptClip);
List<List<IntPoint>> solution = new List<List<IntPoint>>();
clipper.Execute(ClipType.ctUnion, solution, PolyFillType.pftEvenOdd, PolyFillType.pftEvenOdd));
_temporaryPolygon.Clear();
_fullPolygon = solution;

And that’s all for today, I hope you enjoyed it. I don’t have any source code at this time, since this is still coupled with my game engine, but I might get to it sometime in the future when I have more time. Until then!

In part II I’ve shown how to use the DynamicSoundEffectInstance class in order to to continuously play any single sound signal that we were capable of generating. I also wrote a separate article about oscillators (which you should also read if you haven’t done so already) which describes a simple class which takes care of generating some of the most common types of synthesized sounds.

Now it’s time to bring it all together and build a little interactive synthesizer that we can play like a piano on our keyboard keys! But for that, there are two major problems that we need to address:

1. Polyphony - Being able to play more than one sound at the same time.
2. Time - Being able to control where each note starts and stops.

This time I probably won’t be able to cover every bit of code used in the article, but you can download the sample source code at the end and hopefully it should fill any gaps.

Architecture

The architecture of the synthesizer I created for this article is divided into three classes or components. You can see them on the image below:

The Building Blocks

What this diagram represents is that we’ll haves a Synth class with a bunch of Voice objects inside. You can think of a Voice object as being something that encapsulates a single individual sound, and provides some control over it. Unsurprisingly, the number of Voice objects created determine how many sounds the Synth can play at the same time, or in other words its polyphony.

As for generating the sound itself, the Voice object delegates that task to the Oscillator class that I already discussed. In this case, there will be a single Oscillator shared by all voices, which means that every key we press will be making the same type of sound (but at different frequencies).

So, how do we solve the problems of polyphony and time control using this architecture? Simple!

1. In regards to implementing polyphony, what happens is that the Synth class keep a list with all the Voices that are currently being played, and anytime it needs to submit a new audio buffer, it iterates over all active voices passing each of them the same audio buffer. Then it’s up to the Voices to do their own processing on that buffer. The catch is that, unlike what we did in part II, instead of overwriting the buffer with new values, these values will be added to the values that are already there, which effectively mixes all sounds the together!
2. As for having some control over time I’ve just mentioned that all the notes that are playing are put on an active voices list. Because of that, controlling when a note starts or stops is pretty easy! To start a note, just create a Voice and add it to the active voices list. To stop a note, just get the Voice that is associated with it and remove it from the active voices list. In order to allow a Voice to kill itself from inside, I added a boolean IsAlive property to my voices, and when iterating over them on my Synth class, I remove any voice that is already dead from the list. Voila!

The Synth class deserves some special attention because of all the different things it will be doing. Let’s start with an image of its internals:

Synth Architecture

The Audio Engine bit is exactly what was described in part II of the series. It creates a DynamicSoundEffectInstance object plus a couple of buffers, and links everything together, taking care of submiting a new audio buffer when needed. The only big difference is that here the audio engine will loop over all active voices and delegate the task of modifying the audio buffers to them.

The Voices section is responsible for managing all the voices. It uses three different data structures for this:

• Pool of Voices - An array of size Polyphony containing all the voices that will be used by the synthesizer. These will be created when the synthesizer is initialized, and remain constant until the end.
• Active Voices - A list where we store references to all of the voices that are currently playing. This list is picked up by the audio engine and determines which notes we’ll be hearing.
• Free Voices - A stack which will be used to quickly get a voice that is not being used when we ask the synthesizer to play a new note.

Interaction with the synthesizer is done through two methods named NoteOn and NoteOff. The NoteOn method fetches a free voice, initializes it with the frequency corresponding to the note we want to play, and adds it to the active voices list. The NoteOff method simply tells the voice corresponding to that particular note to stop playing.

For this we also need the Note Registry which is a simple dictionary that keeps a connection between each playing note and its respective voice object.

Okay, enough of the architecture. I’ll jump straight into some code now. I’ll try to cover the most important bits, but download the source code at the end of the article and play around with it.

Step 1: The Oscillator Class

I’ll start with the simplest step which is generating the audio signal. I’ve already talked about this subject before. Basically we’ll create a class called Oscillator whose sole purpose is to generate audio signals of different types.

But in comparison with the Oscillator class I described earlier, I made a few modifications this time to make it a bit easier to understand:

• I changed it into a static class with separate methods for each type of wave. Now you don’t even need to create an Oscillator. Instead you only need to call the right method and you’re good to go.
• I also changed the signature of the main function a bit. This time our oscillator will depend on two different parameters (frequency and time) while the signal’s maximum amplitude will remain fixed to 1. Any changes to the signal’s amplitude will come later from our synth.

Here’s the code:

public static class Oscillator
{
    public static float Sine(float frequency, float time)
    {
        return (float) Math.Sin(frequency * time * 2 * Math.PI);
    }

    public static float Square(float frequency, float time)
    {
        return Sine(frequency, time) >= 0 ? 1.0f : -1.0f;
    }

    public static float Sawtooth(float frequency, float time)
    {
        return (float) (2 * (time * frequency - Math.Floor(time * frequency + 0.5)));
    }

    public static float Triangle(float frequency, float time)
    {
        return Math.Abs(Sawtooth(frequency, time)) * 2.0f - 1.0f;
    }
}

public delegate float OscillatorDelegate(float frequency, float time);

You may have noticed that I also declared a delegate with the same signature as these methods. This will be useful to store a reference to one of the oscillator methods, and being able to change them on the fly on my synth class. That’s all for this part, and with this we’ve got the four most common sound waveforms covered. Let’s move to step two then.

Step 2: The Voice Class

Next I’ll be talking about the Voice class. I’ll start with the public interface of the Voice class, and what each of its methods need to do. The comments should be self-explanatory:

// Returns true if voice is still playing, false if it has ended.
bool IsAlive { get; }

// Starts the voice using the chosen frequency
void Start(float frequency);

// Stops voice playback
void Stop();

// Add contribution to the buffer
void Process(float[,] buffer);

Implementing it is not too hard. The Voice class keeps a reference to the parent synth class which is used to access the shared oscillator. Calling Stop will make the voice stop immediatly by setting the IsAlive property to false. The only part that might not be too obvious, is why I multiply the sample by 0.3f before adding it to the buffer. I do so in order to lower its volume a bit and prevent the resulting values to start saturating with just a couple of notes pressed. Still, if you try to play too many notes at the same time, you’re bound to run into distortion problems, unless you start using a more intricate mixing algorithm (instead of just adding the samples together).

public class Voice
{
    public Voice(Synth synth)
    {
        _synth = synth;
    }

    public bool IsAlive { get; private set; }

    public void Start(float frequency)
    {
        _frequency = frequency;
        _time = 0.0f;
        IsAlive = true;
    }

    public void Stop()
    {
        IsAlive = false;
    }

    public void Process(float[,] buffer)
    {
        if (IsAlive)
        {
            int samplesPerBuffer = buffer.GetLength(1);
            for (int i = 0; i < samplesPerBuffer; i++)
            {
                float sample = _synth.Oscillator(_frequency, _time);
                // Important: Use += instead of = !
                buffer[0, i] += sample * 0.3f;
                _time += 1.0f / Synth.SampleRate;
            }
        }
    }

    private float _frequency;
    private float _time;
    private readonly Synth _synth;
}

At the end of this article I’ll show an upgraded version of the Voice class which allows you to add a simple fade-in/fade-out to your voices which gets rid of a few annoying sound problems and makes everything sound better. But for now I’ll move on to the Synth class.

Step 3: The Synth Class

A large part of the Synth class is the low-level audio engine management that I’ve described in part II, so I’ll skip right over those and only talk about the significant changes.

To start with, there’s the four data structures that are used to manage all of the voices. They’re all initialized in the constructor of the class. First, we create an array of voices (sized Polyphony) which will be our voice pool. This is the only point in our application where we will be instantiating Voice objects - after that it’s just references to existing objects being moved around. Next is the stack of free voices, which initially contains all of the voices in the pool. Then there’s a list of active voices, which starts out empty, and is used to keep all the voices that are playing at any given moment. Finally we keep a dictionary linking notes (which in this sample will be an integer value with 0 corresponding to C4, and moving up and down in semi-tones) to the voices they correspond to. This is necessary when we tell the synth to stop playing a note, so that it can delegate the order to the appropriate voice.

public const int Polyphony = 32;
private readonly Voice[] _voicePool;
private readonly List<Voice> _activeVoices;
private readonly Stack<Voice> _freeVoices;
private readonly Dictionary<int, Voice> _noteRegistry;

// On Constructor
_voicePool = new Voice[Polyphony];
for (int i = 0; i < Polyphony; ++i)
{
    _voicePool[i] = new Voice(this);
}
_freeVoices = new Stack<Voice>(_voicePool);
_activeVoices = new List<Voice>();
_noteRegistry = new Dictionary<int, Voice>();

Next in line is our SubmitBuffer method (once again see part II for clarifications). The first big difference is that we need to clear the buffer (fill it completely with zeros) at the top of each submission. This is necessary because our voices won’t be overwriting the buffer like we before but rather adding their values on top of the existing buffer, so wee need an empty plate to start with (think of it as calling the GraphicsDevice.Clear method for graphics). I’ve gone and moved the ConvertBuffer method into a static helper class, but its contents are still the same as before.

private void SubmitBuffer()
{
    ClearWorkingBuffer(); // Fill buffer with zeros
    FillWorkingBuffer();
    SoundHelper.ConvertBuffer(_workingBuffer, _xnaBuffer); // Same method used in part II
    _instance.SubmitBuffer(_xnaBuffer);
}

Where most of the magic starts is inside the FillWorkingBuffer method. Whereas we used to directly write to the buffer inside of that method, now the only thing it does is iterate over all the active voices, calling Process on them, and removing any dead voices at the end of it. Notice how I iterate over the voices in reverse order, in order to be able to remove them without invalidating my for-index.

private void FillWorkingBuffer()
{
    for (int i = _activeVoices.Count - 1; i >= 0; --i)
    {
        Voice voice = _activeVoices[i];
        voice.Process(_workingBuffer);

        if (!voice.IsAlive)
        {
            _activeVoices.RemoveAt(i);
            _freeVoices.Push(voice);
        }
    }
}

And finally we have the two methods that provide interactivity to the synthesizer, NoteOn and NoteOff. When calling NoteOn, the first thing it does is check the note registry to ensure that note is not already registered. If everything goes well, it tries to fetch a free voice from the top of the stack, registers that voice to the current note, initializes the voice with the appropriate frequency, and finally adds it to the active voices list. The SoundHelper.NoteToFrequency method is used to calculate the right frequency for each musical note (with 0 corresponding to C4 and so forth). Check the source code at the end for the implementation.

public void NoteOn(int note)
{
    if (_noteRegistry.ContainsKey(note))
    {
        return;
    }

    Voice freeVoice = GetFreeVoice();

    if (freeVoice == null)
    {
        return;
    }

    _noteRegistry[note] = freeVoice;

    freeVoice.Start(SoundHelper.NoteToFrequency(note));

    _activeVoices.Add(freeVoice);
}

As for stopping a note, it’s a matter of checking the note registry to see if it exists, stopping the corresponding voice and removing the note from the registry.

public void NoteOff(int note)
{
    Voice voice;
    if(_noteRegistry.TryGetValue(note, out voice))
    {
        voice.Stop();
        _noteRegistry.Remove(note);
    }
}

You could stop by now, and you’d have a simple synthesizer ready to be played. All you’d have to do is link each NoteOn and NoteOff call with your keyboard’s KeyDown and KeyUp events and you’d be set. But if you do that, you might notice an annoying little “pop” sound when starting or stopping each note. Keep reading to see how this can be solved.

Optional Step 4: Adding Fade-In and Fade-Out

There’s a problem with abruptly starting or stopping a sound: the sudden loss in amplitude of the signal can easily create a noticeable “pop” sound that ruins the whole experience. You can hear that in the video sample at the end of the article, whenever I disable the fade mechanism.

In order to prevent this, I’m going to modify the Voice class a bit in order to add a controllable fade-in and fade-out period to the sounds. Fading a sound basically means multiplying the value of the sample you generated by a floating point number between 0 and 1. So you need to find a way to start this floating point number at 0, move it until it reaches 1, and then do the opposite motion at the end of the sound.

What surprised me the most about this is that even if the fade duration is really small (like 5ms, which is so small you probably won’t even notice a difference), it will still be enough to get rid of those unwanted noises! Here’s a list of all the changes I did, but check the source code archive at the end for the actual implementation.

Changes to the Voice class:

1. Added FadeInDuration/FadeOutDuration integer variables to my synth which describe the duration of each these phases in samples (so if your sample rate is 44100, then a duration of 44100 will be equivalent to one second).
2. Added three different states to the Voice class: Attack (Fade-In), Sustain (Normal) and Release (Fade-Out). I’ll mention why I chose those names below.
3. Doing Start() now moves to the Attack state if FadeInDuration is not zero.
4. Doing Stop() doesn’t kill the Voice immediatly, but rather changes to the Release state (unless FadeOutDuration is zero).
5. Use a counter inside Process to keep track of time and calculate the correct fading factor (between 0 and 1).
6. Multiply the sample generated by the fading factor to modify it.
7. Killing the Voice is now done inside Process when the Release state ends.

This is actually one of the subjects I want to treat in more detail in a later article. What we’re doing is basically modifying the volume or amplitude of our signal over time. This is something usually referred to as an envelope or ADSR envelope. The names I chose (Attack, Sustain and Release) are related to that subject too! But for now just think of them as being simple fade operations.

Doing these changes has a few interesting implications in our synth. The graphics below describe in general terms how the volume of our sounds vary with time, and help correlate them with the NoteOn and NoteOff commands that we issue. Initially, before adding fading to our Voice class, what we had was something like this:

Without Attack/Release

Each voice was alive only during the time between our NoteOn and NoteOff, and the changes in volume were abrupt. Now with these changes, we have something like this instead:

With Attack/Release

See how the note remains alive for a bit even after the NoteOff command? And finally, there’s a special case I’d like to talk about. What happens if your NoteOff command happens before the attack phase ends? In that case we should skip the sustain phase completely, and start reversing the volume back to 0 from the point it was currently at:

Special Case

Conclusions

And that’s all for today! I hope you enjoyed the series, and feel free to leave any questions or suggestions in the comments section below. As usual, I’ll end the article with a video sample and the source code for you to compile and play around with. Until next time!

Source Code

Note: You can disregard the Piano class. It’s just a hack I threw together to render a simple piano keyboard procedurally.

They say a picture is worth a thousand words, but with parallax scrolling you really have to see it in motion in order to appreciate it. So I’ll start by showing the video sample for this article right away:

There you have it. So, there are actually many different ways to achieve this effect, but I’d like to focus on the two that I find easier to use:

1. Scrolling Textures - Useful for scenes like a starfield or a 2D sky. Somewhat limited beyond that.
2. Camera Modification - General purpose and much more flexible.

Method 1 - Scrolling Textures

If you want to implement parallax scrolling entirely with repeating textures, then the easiest way is to use the technique described in my earlier article called Scrolling Textures in XNA. This is useful for instance to create starfields or 2D skies, but not as useful for games where the background needs to change constantly.

So let’s say you have three textures (texture1, texture2 and texture3) that you want to draw on top of each other and scroll them at different speeds. All you have to do is use the technique I described passing your camera position as your texture’s scroll position, and multiplying each of them by a different constant (e.g. 1.0f would mean full speed, 0.5f half speed, 2.0f double speed, etc.). Example below:

spriteBatch.Begin(SpriteSortMode.Deferred, null, SamplerState.LinearWrap, null, null);
spriteBatch.Draw(texture1, position, new Rectangle(cameraX * 0.5f, cameraY * 0.5f, texture1.Width, texture1.Height), Color.White);
spriteBatch.Draw(texture2, position, new Rectangle(cameraX * 0.8f, cameraY * 0.8f, texture2.Width, texture2.Height), Color.White);
spriteBatch.Draw(texture3, position, new Rectangle(cameraX * 1.0f, cameraY * 1.0f, texture3.Width, texture3.Height), Color.White);
spriteBatch.End();

That’s all there is to it! And in case you didn’t notice, the video sample of my Scrolling Textures in XNA article already had parallax scrolling in it - there are in fact three layers of clouds moving at different speeds - although the effect is very subtle!

Update: I wrote a new article about this topic which lets you use zoom and rotation at the same time as scrolling the background infinitely. Check it out here.**

Method 2 - Camera Modification

Although using scrolling textures is quite easy, it is not applicable to all cases. Most of the time, your game world simply won’t be made up of repeating textures, but you’ll still want to leverage parallax scrolling. Other times you’ll also need the ability to move your camera around, rotate it and zoom it.

A couple years ago, David Amador wrote a popular article on how to create a 2D camera with zoom and rotation using a simple transformation matrix. I’m not going to describe the entire technique again here, so if you don’t already have a matrix based 2D camera implemented, you might want to start there in order to understand the basics.

It turns out that adding parallax scrolling support to a matrix based camera class is extremely simple (the video at the top of the article uses this technique). Here’s the entire camera implementation:

public class Camera
{
    public Camera(Viewport viewport)
    {
        Origin = new Vector2(viewport.Width / 2.0f, viewport.Height / 2.0f);
        Zoom = 1.0f;
    }

    public Vector2 Position { get; set; }
    public Vector2 Origin { get; set; }
    public float Zoom { get; set; }
    public float Rotation { get; set; }

    public Matrix GetViewMatrix(Vector2 parallax)
    {
        // To add parallax, simply multiply it by the position
        return Matrix.CreateTranslation(new Vector3(-Position * parallax, 0.0f)) *
            // The next line has a catch. See note below.
            Matrix.CreateTranslation(new Vector3(-Origin, 0.0f)) *
            Matrix.CreateRotationZ(Rotation) *
            Matrix.CreateScale(Zoom, Zoom, 1) *
            Matrix.CreateTranslation(new Vector3(Origin, 0.0f));
    }
}

I simplified it as much as possible, but the only change made in order to support parallax was to make the GetViewMatrix method require you to specify the desired parallax factor (if you do not wish to have any parallax, simply pass it Vector.One), which is then multiplied by the camera’s position when calculating the view matrix.

The Origin Trick: I made another change in comparison with David Amador’s method (which is more a matter of personal preference) that was include an additional (negative) Origin translation matrix before the Rotation and Zoom matrices (I marked the code with a comment). I prefer doing this because it makes your origin calculations only affect zooming and rotating, but leave translation untouched. That’s because I prefer to set my camera’s position using the top-left corner, but still want to have it rotate and zoom around the center of the screen. This way you can have both at the same time! Also I recommend doing this since it makes some optional calculations (such as limiting the camera’s position or looking at an object) easier to handle, because you won’t have to factor in your origin.

What this means is that, for instance, if you pass it a Vector2(0.5f, 0.5f), objects will appear to move at half of their original speed, and if you pass it a Vector2(2.0f, 2.0f) they’ll appear to move twice as fast. As I mentioned above, passing it Vector.One means that there will be no parallax scrolling applied to that object, and passing it a Vector2.Zero will make the object stay in the same place even as you move the camera around. You can even pass different amounts of parallax for the X and Y axis if you wish!

Usage Scenario

Now, in order to use this class, simply instantiate it passing it your current viewport:

Camera camera = new Camera(GraphicsDevice.Viewport);

And use the GetViewMatrix method to get a transformation matrix for your SpriteBatch:

Vector2 parallax = new Vector2(0.5f);
spriteBatch.Begin(SpriteSortMode.Deferred, null, null, null, null, null, camera.GetViewMatrix(parallax));
spriteBatch.Draw(texture, position, Color.White); // This sprite will appear to move at 50% of the normal speed
spriteBatch.End();

By the way, I’m not worrying about performance optimizations in this article for the sake of simplicity. In practice, you might not want to recalculate the view matrix every frame, and instead cache it and only update it when the camera changes. I’ll leave that exercise up to the reader, but if you need a tip just leave a comment below.

If all you wanted to know was how to implement parallax scrolling, then by now you should already have everything that you need. Below I’ll be describing some more advanced topics, which I believe should be worthwile to know, so I recommend reading to the end!

Optional: Layer Grouping

I find that the easiest way to handle different parallax values it to introduce the concept of layers. A layer is simply a group of game objects that share the same parallax value. Ideally you should be able to create as many layers as you want, set each of their parallax speeds, add your game objects to them, and just tell them to draw themselves. Here’s how I usually do it:

I’ll start by creatig a Layer class which has its own parallax speed, and holds a bunch of Sprite objects. Calling Draw will make all sprites be displayed with the correct parallax.

Here’s a minimal Sprite class implementation (for simplicity I stripped out rotation and scaling information, since it doesn’t serve any purpose in this example):

public struct Sprite
{
    public Texture2D Texture;
    public Vector2 Position;

    public void Draw(SpriteBatch spriteBatch)
    {
        if(Texture != null)
            spriteBatch.Draw(Texture, Position, Color.White);
    }
}

And here’s the Layer class. You need to already have a Camera object created when you create a new Layer, but after that it’s pretty straighforward. The only significant part is the call to SpriteBatch.Begin which relies on the camera to build a view matrix for the appropriate parallax value.

public class Layer
{
    public Layer(Camera camera)
    {
        _camera = camera;
        Parallax = Vector2.One;
        Sprites = new List<Sprite>();
    }

    public Vector2 Parallax { get; set; }
    public List<Sprite> Sprites { get; private set; }

    public void Draw(SpriteBatch spriteBatch)
    {
        spriteBatch.Begin(SpriteSortMode.Deferred, null, null, null, null, null, _camera.GetViewMatrix(Parallax));
        foreach(Sprite sprite in Sprites)
            sprite.Draw(spriteBatch);
        spriteBatch.End();
    }

    private readonly Camera _camera;
}

Usage Scenario

Putting it all together, you could for example setup a scene with 9 different parallax layers. Each of these layers will get a different parallax value, hold one sprite:

_layers = new List<Layer>();

// Create a camera instance and limit its moving range
_camera = new Camera(GraphicsDevice.Viewport) { Limits = new Rectangle(0, 0, 3200, 600) };

// Create 9 layers with parallax ranging from 0% to 100% (only horizontal)
_layers = new List<Layer>
{
    new Layer(_camera) { Parallax = new Vector2(0.0f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.1f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.2f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.3f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.4f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.5f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.6f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(0.8f, 1.0f) },
    new Layer(_camera) { Parallax = new Vector2(1.0f, 1.0f) }
};

// Add one sprite to each layer
_layers[0].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer1") });
_layers[1].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer2") });
_layers[2].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer3") });
_layers[3].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer4") });
_layers[4].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer5") });
_layers[5].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer6") });
_layers[6].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer7") });
_layers[7].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer8") });
_layers[8].Sprites.Add(new Sprite { Texture = Content.Load<Texture2D>("Layer9") });

And draw everything with a simple loop:

foreach (Layer layer in _layers)
    layer.Draw(_spriteBatch);

And if you want to move the camera around, just use the properties exposed by the Camera class. All parallax magic will be done for you behind the scenes!

Frequently Asked Questions

Since I probably won’t be talking about the topic of 2D cameras again, I’ll take the chance to answer a few questions that I often see on the forums:

Q: When moving the camera around while it’s rotated, how do I make it follow the camera’s rotation?

A: You’ll need to do a little math for this. Add this method to your camera class:

public void Move(Vector2 displacement, bool respectRotation = false)
{
    if (respectRotation)
    {
        displacement = Vector2.Transform(displacement, Matrix.CreateRotationZ(-Rotation));
    }

    Position += displacement;
}

Now whenever you want to move in accordance with the camera’s rotation, just set the “respectRotation” parameter to true!

Q: How do I make my camera look at some object, or follow my character?

A: Centering your camera around an object is simply a matter of settings its position to the same as the object, and subtracting half of the screen’s size from it (so you need to store your screen size inside your camera class. For this example I’ll assume that you stored a Viewport object in a member variable called _viewport). Here’s an helper method that will center the camera around some position:

public void LookAt(Vector2 position)
{
    Position = position - new Vector2(_viewport.Width / 2.0f, _viewport.Height / 2.0f);
}

Now if you want to make your camera always follow your character, all you need to do is add something like this to the end of your update loop:

// Updates your character's position
character.Update(gameTime);

// Updates your camera to lock on the character
camera.LookAt(character.Position);

Note: If you didn’t use my origin trick (explained earlier), then you’ll also need to add the camera’s Origin at the end in order to correct the offset.

Q: How do I make the camera stop moving beyond the edges of my level?

A: Okay, this is a bit more complicated, so I’ll go over it in a few separate steps.

First of all, just like the previous question, one thing your limiting code will need to know is how big your viewport is. So start by storing the viewport when you get it in the constructor for later use.

public Camera(Viewport viewport)
{
    _viewport = viewport;
}

private readonly Viewport _viewport;

Next I’ll add a Rectangle property called Limits that lets you choose the extent of what your camera should be able to see. This extent should always be at least as big as the viewport. The question mark (?) after Rectangle makes it possible to assign it a null value. This isn’t normally possible because Rectangle is a value type. I made it this way so that setting Limits to null will disable camera bound limiting altogether. Here’s the code (notice how I validate the rectangle to ensure it’s always bigger than the viewport):

public Rectangle? Limits
{
    get { return _limits; }
    set
    {
        if(value != null)
        {
            // Assign limit but make sure it's always bigger than the viewport
            _limits = new Rectangle
                {
                    X = value.Value.X,
                    Y = value.Value.Y,
                    Width = System.Math.Max(_viewport.Width, value.Value.Width),
                    Height = System.Math.Max(_viewport.Height, value.Value.Height)
                };

            // Validate camera position with new limit
            Position = Position;
        }
        else
        {
            _limits = null;
        }
    }
}

private Rectangle? _limits;

Finally, change your Position property so that its setter checks for the existance of a limit, and clamps the position accordingly. This technique as it stands doesn’t work so well when the camera is scaled or rotated, so I disabled the limiting behavior when the camera is transformed for the sake of correctness. But feel free to try to work around that limitation!

public Vector2 Position
{
    get { return _position; }
    set
    {
        _position = value;

        // If there's a limit set and the camera is not transformed clamp position to limits
        if(Limits != null && Zoom == 1.0f && Rotation == 0.0f)
        {
            _position.X = MathHelper.Clamp(_position.X, Limits.Value.X, Limits.Value.X + Limits.Value.Width - _viewport.Width);
            _position.Y = MathHelper.Clamp(_position.Y, Limits.Value.Y, Limits.Value.Y + Limits.Value.Height - _viewport.Height);
        }
    }
}

That’s it! Now if your level starts at (0,0) and has a size of (8000, 600) you can make sure your camera won’t leave that region by simply doing:

yourCamera.Limits = new Rectangle(0, 0, 8000, 600);

Even better is that if you combine this with the LookAt method described above, you will automatically get the common behavior where the camera seems to stop when your character reaches the borders of the map, and resumes again when you move away from the borders! No additional code needed.

Note: Once again, if you didn’t use the “origin trick” before you’ll also need to add the origin’s X and Y values to the Clamp calculation (inside the Clamp operations, on the min and max parameters).

Update: I wrote another article with more information about this matter. Check it out here: Limiting 2D Camera Movement with Zoom.

Q: How do I find what position in the screen (or world) a certain point in the world (or screen) corresponds to?

This is common for instance when you want to pick objects on the screen with your mouse (screen to world), or to find out which objects are inside of your view (world to screen). If you have parallax scrolling, then this is something you’ll want to do at layer level (since each layer can have a different view matrix), but the general idea is still the same. If you don’t have parallax scrolling, just do this in your camera class instead with the appropriate changes!

public Vector2 WorldToScreen(Vector2 worldPosition)
{
    return Vector2.Transform(worldPosition, camera.GetViewMatrix(parallax));
}

public Vector2 ScreenToWorld(Vector2 screenPosition)
{
    return Vector2.Transform(screenPosition, Matrix.Invert(camera.GetViewMatrix(parallax)));
}

This will get you the correct results, even if your camera is all rotated and scaled.

Conclusions

That’s all for this article. If you have any questions that I didn’t cover here, leave a comment below and I’ll do my best to answer them and update the FAQ accordingly. And as usual, you can find the source code for this article’s sample below. The camera class provided has all of the tips I described implemented, and I also included a SimpleCamera class (although I’m not using it) which is the bare minimum you need to get things working.

Source Code

In general terms, an oscillator is a mechanical or electronic device that generates a signal. The nature of the generated signal depends on how the oscillator is built. In most oscillators used by real synthesizers, the frequency of the signal is controlled by varying the voltage of its input. On the other hand, the amplitude of the generated signal is usually fixed, and has to be modified later using amplifiers or envelopes (something that I’ll talk about some other time).

In our case, since we’re emulating an oscillator in software, we can pretty much do whatever we want. We can specify whatever parameters we need to control the sound, and we’re under no obligiation to mimic how a real oscillator works. So the first think I did was go ahead and add two properties to directly control the amplitude and frequency of the signal.

public double Amplitude = 1.0;
public double Frequency = 440.0;

There. Another thing we need is a way to get the generated signal. In conformance with the interface described in my earlier article, I created a method to give me the amplitude of the signal for any given time:

public double GetSignal(double time);

Finally, I wanted my oscillator to support a few different types of waves. I created an enumeration of all types of waves I needed, and a property to hold what type of wave to use.

public enum WaveformTypes
{
    Sine,
    Triangle,
    Square,
    Sawtooth,
    Pulse,
    Noise
}

public WaveformTypes Waveform = WaveformTypes.Sine;

So, what’s with all these types of waves? Actually, these types of waves are special because they form the basis for the kind of synthesis we’ll be using - subtractive synthesis.

Subtractive Synthesis

There are more than one way to generate sounds. The one we’ll be using is called subtractive synthesis. It’s called this way because we start with a sound that contains all the required harmonic content for the final sound, and then remove any wanted frequencies from the sound with a filter, while shaping the sound’s volume with an envelope. This is usually viewed as a two step process, where each component is either a source (which generates the original sound) or a modifier (which changes the sound to our needs).

Right now we’re only interested in sources, because that’s what our oscillator is. Most sources used in subtractive synthesizers are based around mathematic concepts, such as waves - or as they’re commonly called, waveforms. There are a few types of waveforms present in almost any oscillator used for synthesizers:

Sine

Triangle

Square

Sawtooth

Pulse

Noise

Notes: The last one is actually not a waveform, but rather a random signal which contains a constantly changing mix of all frequencies. It is also frequently known as white noise. Another thing to notice is that the only difference between a square waveform and a pulse waveform is that the square waveform “switches” side exactly in the middle of its period, while the pulse wave can change earlier or later than that (this is controlled by a parameter known as duty cycle or pulse width). In fact a square waveform is simply a pulse waveform with a duty cycle of 50%.

Each of them sounds slightly different from the others, not only because of their shape, but because of the harmonics they contain. The theory behind it is a little complicated, so for now I would just experiment with each of them in order for the ear to become used to how they sound. You’ll realize for instance, that the triangle waveform sounds very soothing, while the sawtooth is much harsher.

Implementation

For the implementation of these waveforms, I simply looked up their mathematical formulas on wikipedia and translated them to code. Here’s the complete GetSignal method.

public double GetSignal(double time)
{
    switch (Waveform)
    {
        case WaveType.Sine:
        {
            return Math.Sin(Frequency * time * 2 * Math.PI) * Amplitude;
        }
        case WaveType.Triangle:
        {
            return Math.Abs(2 * (time * Frequency - Math.Floor(time * Frequency + 0.5))) * Amplitude * 2 - Amplitude;
        }
        case WaveType.Square:
        {
            return Math.Sin(Frequency * time * 2 * Math.PI) >= 0 ? Amplitude : -Amplitude;
        }
        case WaveType.Sawtooth:
        {
            return 2 * (time * Frequency - Math.Floor(time * Frequency + 0.5)) * Amplitude;
        }
        case WaveType.Pulse:
        {
            double period = 1.0 / Frequency;
            double timeModulusPeriod = time - Math.Floor(time / period) * period;
            double phase = timeModulusPeriod / period;
            if (phase <= DutyCycle)
                return Amplitude;
            else
                return -Amplitude;
        }
        case WaveType.Noise:
        {
            return (Rng.NextDouble() - Rng.NextDouble()) * Amplitude;
        }
    }
}

And I also added a parameter for controlling the duty cycle of pulse waves, which should always stay between 0 and 1:

public double DutyCycle = 0.5;

And a random number generation for generating the noise signal:

private static readonly Random Rng = new Random();

Usage

If you’ve looked at the sample source code of the previous article you can see how the oscillator class was used. Basically I created two Oscillator instances, one for the left and one for the right channel. When reading input from the keyboard I simply changed the oscillator parameters, and when generating the sound samples, I delegated that to the corresponding GetSignal method of the Oscillator class. That’s it!

Continue To Part III

This time I’ll be covering the DynamicSoundEffectInstance class and how to put some of the concepts we discussed earlier into practice. Therefore (if you haven’t done so already) I recommend that you read part I before proceeding. I’d also like to warn you that this article will be quite long. The good news is that this is the hardest part for a beginner to overcome, and future articles on this series will probably be smaller and in general easier to grasp.

By the way, there’s a video sample at the end of the article in case you’d like to see it in action before you start reading!

Summary

I’ll start with a summary of everything that has been discussed already in part I:

• Sound is the way our brain interprets the pressure oscillations that reach our eardrums.
• These pressure oscillations can be mathematically represented as waves.
• The wave’s basic relation to sound is that amplitude equals volume, frequency equals pitch and shape equals tone.
• A wave can be implemented (in code) as any regular function that returns an amplitude given a specific time input (e.g. double wave(double time)).
• Even though a wave is a continuous function you can approximate it by sampling only a “few” points along the way.

Introduction

Using this knowledge to generate our own sounds in XNA 4.0 is possible because of a new class called DynamicSoundEffectInstance which exposes all of the necessary low-level access. In theory, the whole process is pretty straightforward: you create some function that represents the sound wave you’d like to playback, read (or sample) enough values from that function to get a decent representation, and feed all of them to XNA which routes them through the low-level audio engine that takes care of the rest. In practice however, it’s not that easy because of a few important details that you will have to know in order to use the class properly:

1. (Configuration) The DynamicSoundEffectInstance class is flexible enough to support different sample rates and multiple channel configurations. You’ll need to understand these concepts and set the parameters accordingly.
2. (Format) The DynamicSoundEffectInstance class expects the data you submit to be presented in a very specific way. As we’ll see, this format is not exactly the easiest to work with when doing our wave calculations.
3. (Chunks) Even if you have a large amount of sound data, you (most likely) won’t be submitting all of it at once, since that would be slow. Instead, in order to enable real-time behavior you’ll be dividing your data in small chunks and submitting them one at a time.

And I’ll admit that some of these details can be pretty confusing if you’ve never seen anything like this before. It did take me some time before finally getting that “a-ha” moment. Therefore, let’s take it one step at a time. I’ll describe each step you need to implement in order to use the DynamicSoundEffectInstance class, and present my solutions to the problems encountered during the way. Let’s start!

The Steps

In order to create and use the DynamicSoundEffectInstance class, you’ll need to follow these four steps:

1. Create and configure a DynamicSoundEffectInstance object.
2. Create a buffer to hold your audio samples.
3. Fill the buffer with meaningful values.
4. Submit the buffer to the DynamicSoundEffectInstance object when needed.

Step 1: Create DynamicSoundEffectInstance

If you’ve used XNA’s SoundEffect API before, then you might already be familiar with the SoundEffect and SoundEffectInstance classes. When using the content pipeline to load sound effects for your game, you receive a SoundEffect object that can be used to playback that sound. You can also use the SoundEffect.CreateInstance() method to create single instances of that SoundEffect for individual control. Either way, when loading a sound resource through the content pipeline, all of its data is automatically copied into the object’s audio buffer, and the sound engine only needs to start playing it when requested.

On the other hand, the DynamicSoundEffectInstance class (which inherits from SoundEffectInstance) is created with an empty audio buffer, and instead allows you to fill it with data. But since you’re the one providing it with all of the data, there are some extra decisions you need to take on how the data should be interpreted. Some of these parameters are already fixed for you by the framework while others are selected when creating the DynamicSoundEffectInstance object. Either way it’s important to know exactly what they mean, so I’ll describe them one by one.

Let’s start by taking a look at DynamicSoundEffectInstance’s constructor:

public DynamicSoundEffectInstance (int sampleRate, AudioChannels channels)

As you can see, there are two parameters that you need to pass to the constructor: the sample rate and the audio channels count.

Sample Rate

As explained in part I of this article, the sample rate describes how many audio samples (remember, a sample is exactly one value taken from our wave function) will be processed every second (for each audio channel). Having a larger sample rate is desirable because it means that you can better represent fast variations in the sound and higher frequencies. The XNA Framework lets you select a sample rate ranging from 8.000 Hz to 48.000 Hz. Audio CD’s use a sample rate of 44.100 Hz so if you’re creating a music application you’ll probably want to use at least that for best results.

Audio Channels Count

You’ve probably heard the words Stereo and Mono before. The main difference between them is that with Mono (1 audio channel), even if you have a pair of speakers, they will both emit the same sound, whereas with Stereo (2 audio channels) you have the freedom to specify different sounds for each speaker (left and right). It’s natural then that with Stereo you’ll need to hold twice the amount of information. The XNA Framework currently supports two possible configurations: AudioChannels.Mono (1 audio channel) and AudioChannels.Stereo (2 audio channels).

Besides these two parameters, there are also a couple things that you do not have control over (because XNA enforces them for you), but should still understand:

Bit Depth

__The bit depth describes how many bits of precision will be used to encode each sample. A larger bit depth implies that you’ll have more precision when dealing with small variations of amplitude in your wave, and get a digital representation that is closer to the original analog wave. When working with waves you’ll normally be generating floating-point samples (usually between -1.0 and 1.0). The problem is that XNA’s audio engine does not work with floating-samples directly. Instead, it works with a fixed bit depth of 16 bits/sample (signed). So basically, you’ll need to convert each of your floating-point samples (in the -1.0 to 1.0 range) to a short (signed 16-bit integer) sample (in the short.MinValue to short.MaxValue) range before feeding them to XNA.

Audio Format

We’ve just seen that XNA forces a signed 16-bit integer bit depth. That’s only one part of the audio format, namely the one which specifies how each audio sample is represented. But we’ve also seen that by choosing a Stereo configuration you can have two audio channels playing simultaneously. The samples for each audio channel will have to be packed into the same buffer, so how are they organized in relation to each other? Well, when you have only one audio channel, each sample is simply stored in the buffer in same the order they appear. But if you have two audio channels, then it’s a different story. Instead, you’ll have to interleave one audio sample from the left channel with one audio sample from the right channel, and repeat until there are no more samples left (i.e. LRLRLRLRLR etc.). Naturally, you’ll also be needing a buffer with twice the size in order to fit all the samples there. I’ll touch this subject again when describing how to create the audio buffers.

Sample Rate (Horizontal) and Bit Depth (Vertical)

Okay, now that you understand most of the necessary concepts, let’s start putting it to practice and create that DynamicSoundEffectInstance object. I’ll be working with a sample rate of 44.100 Hz and Stereo audio for the rest of this article, since it covers most of the edge cases. Here’s the code for this step:

// On your class
private const int SampleRate = 44100;
private const int ChannelsCount = 2;
private DynamicSoundEffectInstance _instance;

// On LoadContent
_instance = new DynamicSoundEffectInstance(SampleRate, (ChannelsCount == 2) ? AudioChannels.Stereo : AudioChannels.Mono);

Step 2: Create Audio Buffers

This is possibly the hardest step to understand in the whole process, because it depends on most of the concepts we’ve just seen. Let’s start from the beginning. A buffer is simply an array of values, and you’ll need to create one big enough to hold the audio data that will be passed to audio engine. But how big should it be, and in what format should it be created?

Size

The first step is to decide how many audio samples (per channel) do you want to submit to XNA at a time (i.e. on each submit operation). To avoid confusion with the physical size of the buffers we’ll be creating, I’ll call this amount “SamplesPerBuffer” from now on. This is an arbitrary value, so you can choose whatever you like, but depending on your choice, there are a few implications. Depending on the size you choose, the audio engine could become unstable, and you’ll have drops in the audio signal and glitches. Also depending on the size you choose, the latency of your signal will change, which in pratical terms determines how fast your application will be able to react to user input. The rules of thumb are:

Big Buffer = More Stable (Good), More Latency (Bad)
Small Buffer = Less Stable (Bad), Less Latency (Good)

Ideally you’ll want to choose a value that is small enough for latency to be near real-time, but large enough not to hear any sound glitches. I usually start around 3000 and then tweak it to my needs. On my Windows machine I can get it down to about 1000 or less and it still works perfectly.

Format

So, we’ve decided that each buffer should hold, for instance, 3000 audio samples (as I said, experiment with this value). Now what format should our buffer have? That’s where it gets tricky. The thing is, the format that is more convenient for doing wave calculations is not the same that XNA expects to receive.

In order to ease this step, I like to create two different buffers: one to do all of our wave math calculations (which I’ll call that the “Working Buffer”), and a different one to actually comunicate with XNA (which I’ll call the “XNA Buffer”). I’ll provide a method that will convert samples from one format to the other so you won’t have to worry about XNA’s internal format at all. Now on to our buffer formats:

Working Buffer Format

1. Our wave function outputs floating-point samples.
2. Each audio channel will be generated and stored separatedly.

So we’ll create a bi-dimensional floating-point array depending on the number of channels, like so:

Working Buffer = float[ChannelsCount, SamplesPerBuffer]

XNA Buffer Format

1. XNA expects each sample to be a signed 16-bit integer (or short).
2. XNA expects both audio channels to be passed together, in an interleaved fashion (LRLRLRLRLR).
3. Important: Even though samples are expected to be 16-bit (short), for compatibility reasons XNA accepts its buffer as an array of bytes (8-bit)! This means that you’ll have to encode each of your 16-bit (short) samples as two consecutive 8-bit (byte) samples instead.

Point 3 is very important because it means that you’ll have to multiply the whole size of your buffer by 2 in order to hold all the necessary bytes (which will be 2 bytes for each sample). Since everything will be stored in a row, we use a uni-dimensional array, like so:

XNA Buffer = byte[ChannelsCount * SamplesPerBuffer * 2]
(where 2 is the number of bytes per sample)

I created this image that should help you visualize better how the two buffers are related. The buffer in the middle is never really created, but serves to illustrate how we need to convert our samples from floats/doubles to shorts, before splitting them up into bytes (click to see in full size).

Audio Buffers

That’s all there is to it. As long as you create these arrays with the right size and format, you should be okay. Here’s the code:

// On your class
public const int SamplesPerBuffer = 3000;
private float[,] _workingBuffer;
private byte[] _xnaBuffer;

// On LoadContent, after creating the DynamicSoundEffectInstance object
_workingBuffer = new float[ChannelsCount, SamplesPerBuffer];
const int bytesPerSample = 2;
_xnaBuffer = new byte[ChannelsCount * SamplesPerBuffer * bytesPerSample];

And the method I mentioned before that converts the contents of the working buffer into the xna buffer is (just add it to your class, it’s an utility method):

private static void ConvertBuffer(float[,] from, byte[] to)
{
    const int bytesPerSample = 2;
    int channels = from.GetLength(0);
    int samplesPerBuffer = from.GetLength(1);

    // Make sure the buffer sizes are correct
    System.Diagnostics.Debug.Assert(to.Length == samplesPerBuffer * channels * bytesPerSample, "Buffer sizes are mismatched.");

    for (int i = 0; i < samplesPerBuffer; i++)
    {
        for (int c = 0; c < channels; c++)
        {
            // First clamp the value to the [-1.0..1.0] range
            float floatSample = MathHelper.Clamp(from1, -1.0f, 1.0f);

            // Convert it to the 16 bit [short.MinValue..short.MaxValue] range
            short shortSample = (short) (floatSample >= 0.0f ? floatSample * short.MaxValue : floatSample * short.MinValue * -1);

            // Calculate the right index based on the PCM format of interleaved samples per channel [L-R-L-R]
            int index = i * channels * bytesPerSample + c * bytesPerSample;

            // Store the 16 bit sample as two consecutive 8 bit values in the buffer with regard to endian-ness
            if (!BitConverter.IsLittleEndian)
            {
                to[index] = (byte)(shortSample >> 8);
                to[index + 1] = (byte)shortSample;
            }
            else
            {
                to[index] = (byte)shortSample;
                to[index + 1] = (byte)(shortSample >> 8);
            }
        }
    }
}

The method is heavily commented in order to explain what’s going on, but if you have any question about it, feel free to leave it in the comment section below.

Step 3: Fill Buffers

Now that you’ve gotten here, the rest is actually quite easy! Remember that any calculations you do will be stored in the Working Buffer. Then at the end, you simply use the ConvertBuffer method above to copy it to the XNA buffer in the right format. For the sake of keeping it easy, we’ll be using a simple sine wave function that is described like this:

double SineWave(double time, double frequency)
{
    return Math.Sin(time * 2 * Math.PI * frequency);
}

Then we’ll be filling our working buffer, such that the left and right audio channels are playing sine waves of different frequencies (in order to clearly demonstrate the effects of stereo audio). Generating waves of different frequencies is pretty straightforward, since you just need to pass a different “frequency” parameter to the function. But what value do we pass to the “time” parameter? We need a way to know, for each individual sample we’re generating, at what time that sample corresponds to. For that we need to create some sort of timer.

First, create a variable to count how much time has passed so far:

// On your class
private double _time = 0.0;

But instead of incrementing it in our Update method as you’re probably used to, we should increment it whenever we advance to the next sample during generation. Calculating how much we should advance our time variable is also easy. We know, for instance, that if we have a SampleRate of 44100 Hz, this means that there will be 44100 samples in a second. Therefore, we can also deduce that each sample has a duration of 1 / 44100 seconds! So basically all we have to do is advance our time variable in (1 / SampleRate) increments, and our variable will always hold a very accurate representation of time (just check the example code below).

So let’s put this all together and create a FillWorkingBuffer() method that fills our buffer with values from the SineWave function above, while automatically counting the time for us. Note that this function is generating a 440 Hz sine wave for the left channel, and a 220 Hz sine wave for the right channel:

private void FillWorkingBuffer()
{
    for (int i = 0; i < SamplesPerBuffer; i++)
    {
        // Here is where you sample your wave function
        _workingBuffer[0, i] = (float) SineWave(_time, 440); // Left Channel
        _workingBuffer[1, i] = (float) SineWave(_time, 220); // Right Channel

        // Advance time passed since beginning
        // Since the amount of samples in a second equals the chosen SampleRate
        // Then each sample should advance the time by 1 / SampleRate
        _time += 1.0 / SampleRate;
    }
}

Also don’t forget to call ConvertBuffer() at the end in order to convert your working buffer into XNA’s format!

Step 4: Submit Buffer

All that’s left is to call Play() on our DynamicSoundEffectInstance object and start submitting buffers to it. Submitting the buffer itself is pretty trivial, all you need to do is call:

DynamicSoundEffectInstance.SubmitBuffer(byte[] buffer)

Since filling, converting and submiting a buffer is usually done in sequence, let’s group all three operations in a function for ease of use:

private void SubmitBuffer()
{
    FillWorkingBuffer();
    ConvertBuffer(_workingBuffer, _xnaBuffer);
    _instance.SubmitBuffer(_xnaBuffer);
}

We’re almost there. There’s only one question remaining! How do I know when it’s time to submit a new buffer? There are two different ways to accomplish this:

1. Hook the DynamicSoundEffectInstance.BufferNeeded event and submit it there.
2. Poll DynamicSoundEffectInstance.PendingBufferCount in your update loop and decide yourself when to submit.

I prefer way number 2 because it allows me to keep a consistent amount of buffers in reserve, which prevents audio glitches and dropouts. I tried doing that with the BufferNeeded event (method 1), and when monitoring my PendingBufferCount value, it would keep fluctuating around every second, whereas with method 2 it stays stable. Furthermore, it’s really easy! All you need to do is add this to your Update method:

// On Update
while(_instance.PendingBufferCount < 3)
    SubmitBuffer();

At last, simply call Play() on your DynamicSoundEffectInstance object after creating the buffers and preparing everything, in order to start playback of your sound!

// At the end of LoadContent
_instance.Play();

Sample

Here’s a sample video and source code showing everything described in this article put into practice. You’ll probably notice that there’s many different types of waves being generated (sine, square, pulse, triangle, sawtooth, noise). Since this article became big enough as it is, I’ll be covering those in a separate article dedicated to describing the Oscillator class I’m using on the sample. An oscillator is basically a device that creates wave signals for you. Don’t worry, it’s not complicated at all compared to what we already did.

Source Code

Conclusions

That’s it! I know it was long, but by now you hopefully have an annoying buzzing sound playing on your speakers. There’s still many things missing though, most notably:

• You can’t play more than one sound at once (like a piano, i.e. we lack polyphony).
• You can’t control the note’s durations (i.e. when they start and stop, etc)

These are the two questions that I’ll address in part III of the series, which will show you how to create a little playable “piano” with dynamically generated audio. It will also describe how to start shaping all of this code into a framework so that we can have more flexibility to add new sounds, effects and other goodies in the future. Meanwhile, be sure to also check my article about oscillators in sound synthesis. Until then!

Continue To Part III