Chapters
Chapters

Chapters

Hide chapters

Metal by Tutorials

Second Edition · iOS 13 · Swift 5.1 · Xcode 11

Before You Begin

Section 0: 3 chapters
Show chapters Hide chapters

Section I: The Player

Section 1: 8 chapters
Show chapters Hide chapters

Section II: The Scene

Section 2: 7 chapters
Show chapters Hide chapters

Section III: The Effects

Section 3: 10 chapters
Show chapters Hide chapters

4. Coordinate Spaces
Written by Caroline Begbie

Heads up... You're reading this book for free, with parts of this chapter shown beyond this point as scrambled text.

You can unlock the rest of this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now

Everything you see on your device’s screen, from the characters you type to the digital art you admire, is all math under the surface.

We’d all love to be math geniuses, but some of us lost the opportunity early in life. Fortunately, to use math, you don’t always have to know what’s under the hood. In this chapter, you’re going to become a matrix master, and you’ll learn what matrices can do for you and how to manipulate them painlessly.

You’ll start by learning how to move, scale and rotate a triangle using matrices. Once you’ve mastered one triangle, it’s a cinch to rotate thousands of triangles at once. Then, you’ll upgrade your train project from the previous chapter and understand why you use matrices and what coordinate spaces are.

Transformations

In this picture, the vector image editor, Affinity Designer, was used to translate, scale and rotate a cat through a series of affine transformations.

Instead of individually working out each position, Affinity Designer creates a transformation matrix that holds the combination of the transformations. It then applies the transformation to each element.

Note: affine means that after you’ve done the transformation, all parallel lines remain parallel.

Scaling cats is difficult because they can bite, so instead, you’ll try out translating, scaling and rotating vertices in a Playground.

Translation

Open the starter project playground located in the starter folder for this chapter. Run the playground, and you’ll get a blank cream screen. The starter playground is set up ready for drawing but has nothing to draw yet.

Vertex function

Inside the Resources folder for the playground, open Shaders.metal.

struct VertexOut {
  float4 position [[position]];
  float point_size [[point_size]];
};

// 1
vertex VertexOut 
       vertex_main(constant float3 *vertices [[buffer(0)]],
// 2
  uint id [[vertex_id]])
{
  // 3
  VertexOut vertex_out {
    .position = float4(vertices[id], 1),
    // 4
    .point_size = 20.0
  };
  return vertex_out;
}

Fragment function

Now replace the fragment function with this code:

fragment float4 
         fragment_main(constant float4 &color [[buffer(0)]]) 
{
  return color;
}

Set up the data

Back in the main playground page, you’ll draw two vertices. You’ll hold one vertex point in an array and create two buffers: one containing the original point, and one containing the translated point.

// drawing code here

var vertices: [float3] = [[0, 0, 0.5]]

let originalBuffer = device.makeBuffer(bytes: &vertices, 
      length: MemoryLayout<float3>.stride * vertices.count, 
      options: [])

renderEncoder.setVertexBuffer(originalBuffer, 
                              offset: 0, index: 0)
renderEncoder.setFragmentBytes(&lightGrayColor,
                    length: MemoryLayout<float4>.stride, 
                    index: 0)

Draw

Do the draw call using the primitive type .point:

renderEncoder.drawPrimitives(type: .point, vertexStart: 0, 
                             vertexCount: vertices.count)


vertices[0] += [0.3, -0.4, 0]
var transformedBuffer = device.makeBuffer(bytes: &vertices, 
       length: MemoryLayout<float3>.stride * vertices.count, 
       options: [])

renderEncoder.setVertexBuffer(transformedBuffer, 
                              offset: 0, index: 0)
renderEncoder.setFragmentBytes(&redColor,
                       length: MemoryLayout<float4>.stride, 
                       index: 0)
renderEncoder.drawPrimitives(type: .point, vertexStart: 0, 
                       vertexCount: vertices.count)

Vectors and matrices

You can describe your previous translation as a displacement vector of [0.3, -0.4, 0]. You moved the vertex 0.3 units in the x-direction, and -0.4 in the y-direction from its starting position.

var matrix = matrix_identity_float4x4

vertices[0] += [0.3, -0.4, 0]

matrix.columns.3 = [0.3, -0.4, 0, 1]

vertices = vertices.map {
  let vertex = matrix * float4($0, 1)
  return [vertex.x, vertex.y, vertex.z]
}

Matrices on the GPU

You may have noticed that this vertex processing code is taking place on the CPU. This is serial processing, which is much more inefficient compared to parallel processing. There’s another place where each vertex is being processed — the GPU. You can pass the GPU your transformation matrix and multiply every vertex in the vertices array by the matrix in the vertex shader. The GPU is optimized for matrix calculation.

vertices = vertices.map {
  let vertex = matrix * float4($0, 1)
  return [vertex.x, vertex.y, vertex.z]
}

renderEncoder.setVertexBytes(&matrix, 
     length: MemoryLayout<float4x4>.stride, index: 1)

vertex VertexOut 
        vertex_main(constant float3 *vertices [[buffer(0)]],
                    constant float4x4 &matrix [[buffer(1)]],
                    uint id [[vertex_id]]) 

.position = float4(vertices[id], 1),

.position = matrix * float4(vertices[id], 1),

renderEncoder.setVertexBytes(&matrix, 
           length: MemoryLayout<float4x4>.stride, index: 1)

Scaling

Translating a single vertex is useful, but you’ll want to scale and rotate your models to fit inside your scene.

var vertices: [float3] = [[0, 0, 0.5]]

var vertices: [float3] = [
  [-0.7,  0.8,   1],
  [-0.7, -0.4,   1],
  [ 0.4,  0.2,   1]
]

renderEncoder.drawPrimitives(type: .point, vertexStart: 0, 
                             vertexCount: vertices.count)

renderEncoder.drawPrimitives(type: .triangle, vertexStart: 0, 
                             vertexCount: vertices.count)

matrix.columns.3 = [0.3, -0.4, 0, 1]

let scaleX: Float = 1.2
let scaleY: Float = 0.5
matrix = float4x4(
  [scaleX, 0, 0, 0],
  [0, scaleY, 0, 0],
  [0,      0, 1, 0],
  [0,      0, 0, 1]
)

Rotation

You perform rotation in a similar way to scaling. Replace:

let scaleX: Float = 1.2
let scaleY: Float = 0.5
matrix = float4x4(
  [scaleX, 0, 0, 0],
  [0, scaleY, 0, 0],
  [0,      0, 1, 0],
  [0,      0, 0, 1]
)

let angle = Float.pi / 2.0
matrix.columns.0 = [cos(angle), -sin(angle), 0, 0]
matrix.columns.1 = [sin(angle), cos(angle), 0, 0]

Matrix concatenation

You may want the rotation to take place around a point other than the world origin. You’ll now rotate the triangle around its right-most point.

matrix.columns.0 = [cos(angle), -sin(angle), 0, 0]
matrix.columns.1 = [sin(angle), cos(angle), 0, 0]

var distanceVector = float4(vertices.last!.x,
                            vertices.last!.y,
                            vertices.last!.z, 1)
var translate = matrix_identity_float4x4
translate.columns.3 = distanceVector
var rotate = matrix_identity_float4x4
rotate.columns.0 = [cos(angle), -sin(angle), 0, 0]
rotate.columns.1 = [sin(angle), cos(angle), 0, 0]

matrix = translate.inverse 

matrix = rotate * translate.inverse

matrix = translate * rotate * translate.inverse

Coordinate spaces

Now that you know about matrices, you’ll be able to convert models and entire scenes between different coordinate spaces. Coordinate spaces map different coordinate systems, and just by multiplying a vertex by a particular matrix, you convert the vertex to a different space.

Object space

You may be familiar with Cartesian coordinates from your graphing days. This image is a 2D grid showing possible vertices mapped out in Cartesian coordinates.

World space

In the following picture, the direction arrows mark the origin. This is the center of world space at (0, 0, 0). In world space, the dog is at (1, 0, 1) and the cat is at (-1, 0, -2).

Camera space

For the dog, the center of his universe is the person holding the camera behind the picture. In camera space, the camera is at (0, 0, 0), and the dog is approximately at (-3, -2, 7). When the camera moves, it stays at (0, 0, 0), but the positions of the dog and cat move relative to the camera.

Clip space

The main reason for doing all this math is to turn a three-dimensional scene with perspective into a two-dimensional scene. Clip space is a cube that is ready for flattening.

NDC space

Projection into clip space creates a half cube of w size. During rasterization, the GPU will convert the w into normalized coordinate points between -1 and 1 for the x-axis and y-axis and 0 and 1 for the z-axis.

Screen space

Now that the GPU has a normalized cube, it will flatten clip space and convert into screen coordinates ready to display on the device’s screen.

Converting between spaces

You may have already guessed it: you use transformation matrices to convert from one space to another.

Coordinate systems

Different graphics APIs use different systems. You already found out that Metal’s NDC (Normalized Device Coordinates) use 0 to 1 on the z-axis. You may already be familiar with OpenGL, which uses 1 to -1 on the z-axis.

Upgrade the engine

Open the starter app for this chapter, named Matrices. This app is the same as at the end of the previous chapter, with the addition of MathLibrary.swift. This utility file contains methods that are extensions on float4x4 for creating the translation, scale and rotation matrices that you created in your 3DTransforms playground.

Uniforms

Constant values that are the same across all vertices or fragments are generally referred to as uniforms. You’ll create a uniform struct to hold the conversion matrices and then apply them to every vertex.

#import <simd/simd.h>

Model matrix

Add the uniforms struct to Common.h:

typedef struct {
  matrix_float4x4 modelMatrix;
  matrix_float4x4 viewMatrix;
  matrix_float4x4 projectionMatrix;
} Uniforms;

var uniforms = Uniforms()

let translation = float4x4(translation: [0, 0.3, 0])
let rotation = 
    float4x4(rotation: [0, 0, Float(45).degreesToRadians])
uniforms.modelMatrix = translation * rotation

timer += 0.05
var currentTime: Float = sin(timer)
renderEncoder.setVertexBytes(&currentTime, 
             length: MemoryLayout<Float>.stride, index: 1)

renderEncoder.setVertexBytes(&uniforms, 
             length: MemoryLayout<Uniforms>.stride, index: 1)

#import "Common.h"

vertex float4 vertex_main(const VertexIn vertexIn [[stage_in]],
                    constant Uniforms &uniforms [[buffer(1)]])
{
  float4 position = uniforms.modelMatrix * vertexIn.position;
  return position;
}

View matrix

To convert between world space and camera space, you’ll set a view matrix. Depending on how you want to move the camera in your app, you can construct the view matrix appropriately. The view matrix you’ll create here is a simple one that is best for FPS (First Person Shooter) style games.

uniforms.viewMatrix = float4x4(translation: [0.8, 0, 0]).inverse

float4 position = uniforms.modelMatrix * vertexIn.position;

float4 position = uniforms.viewMatrix * uniforms.modelMatrix 
                      * vertexIn.position;


renderEncoder.setVertexBytes(&uniforms, 
       length: MemoryLayout<Uniforms>.stride, index: 1)

timer += 0.05
uniforms.viewMatrix = float4x4.identity()
uniforms.modelMatrix = float4x4(rotationY: sin(timer))

Projection

So far you haven’t applied any perspective to your render. Perspective is where close objects appear bigger than objects that are farther away.

Projection Matrix

Open Renderer.swift, and at the end of init(metalView:) set up the projection matrix:

let aspect = Float(metalView.bounds.width) / Float(metalView.bounds.height)
let projectionMatrix =
  float4x4(projectionFov: Float(45).degreesToRadians,
           near: 0.1,
           far: 100,
           aspect: aspect)
uniforms.projectionMatrix = projectionMatrix

float4 position = 
      uniforms.projectionMatrix * uniforms.viewMatrix
      * uniforms.modelMatrix * vertexIn.position;

timer += 0.05
uniforms.viewMatrix = float4x4.identity()
uniforms.modelMatrix = float4x4(rotationY: sin(timer))

uniforms.viewMatrix = float4x4(translation: [0, 0, -3]).inverse

let rotation = 
    float4x4(rotation: [0, 0, Float(45).degreesToRadians])

let rotation = 
    float4x4(rotation: [0, Float(45).degreesToRadians, 0])

Perspective divide

Now that you’ve converted your vertices from object space through world space through camera space to clip space, the GPU takes over to convert to NDC coordinates (that’s -1 to 1 in the x and y directions and 0 to 1 in the z direction). The ultimate aim is to scale all the vertices from clip space into NDC space, and by using the fourth w component, this becomes easy.

NDC to screen

Lastly, the GPU converts from normalized coordinates to whatever the device screen size is. You may already have done something like this at some time in your career when converting between normalized coordinates and screen coordinates.

converted.x = point.x * screenWidth/2  + screenWidth/2
converted.y = point.y * screenHeight/2 + screenHeight/2

converted = matrix * point

Update screen dimensions

Currently, when you rotate your iOS device or rescale the macOS window, the train stretches with the size of the window. You’ll need to update the aspect ratio for the projection matrix whenever this happens. Fortunately MTKViewDelegate gives you a method whenever the view’s drawable size changes.

let aspect = Float(view.bounds.width) / Float(view.bounds.height)
let projectionMatrix =
  float4x4(projectionFov: Float(70).degreesToRadians,
           near: 0.001,
           far: 100,
           aspect: aspect)
uniforms.projectionMatrix = projectionMatrix

mtkView(metalView, 
        drawableSizeWillChange: metalView.bounds.size)

Where to go from here?

You’ve covered a lot of mathematical concepts without diving too far into the underlying mathematical principles. To get started in computer graphics, you can fill your transform matrices and continue multiplying them at the usual times, but to be sufficiently creative, you will need to understand some linear algebra.

3. The Rendering Pipeline
5. Lighting Fundamentals
Have a technical question? Want to report a bug? You can ask questions and report bugs to the book authors in our official book forum here.
© 2024 Kodeco Inc.

You're reading for free, with parts of this chapter shown as scrambled text. Unlock this book, and our entire catalogue of books and videos, with a Kodeco Personal Plan.

Unlock now